2026-02-24T17:08:19.422 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-24T17:08:19.436 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T17:08:19.514 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial/68438 branch: wip-yuri8-testing-2026-02-23-2128 description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '68438' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri8-testing-2026-02-23-2128 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 workunit: branch: wip-yuri8-testing-2026-02-23-2128 sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6922 sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri8-testing-2026-02-23-2128 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 targets: trial066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFqCbWjX1l4V+ppboKjfZbBbXXYZMeuoJTxTvZdE1YSbxgv4W68C3/iA8gz3c3kontzrxVSyzCxre3pR5+VtJaE= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-24_16:03:54 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-24T17:08:19.514 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa; will attempt to use it 2026-02-24T17:08:19.514 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_618c318a7a0b287ab5aeff207b7bf1d73ef45044/qa/tasks 2026-02-24T17:08:19.514 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-24T17:08:19.517 INFO:teuthology.task.internal:Checking packages... 2026-02-24T17:08:19.517 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '618c318a7a0b287ab5aeff207b7bf1d73ef45044' 2026-02-24T17:08:19.517 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-24T17:08:19.517 INFO:teuthology.packaging:ref: None 2026-02-24T17:08:19.517 INFO:teuthology.packaging:tag: None 2026-02-24T17:08:19.517 INFO:teuthology.packaging:branch: wip-yuri8-testing-2026-02-23-2128 2026-02-24T17:08:19.517 INFO:teuthology.packaging:sha1: 618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T17:08:19.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri8-testing-2026-02-23-2128 2026-02-24T17:08:19.640 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5507-g618c318a-1jammy 2026-02-24T17:08:19.641 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-24T17:08:19.653 INFO:teuthology.task.internal:no buildpackages task found 2026-02-24T17:08:19.653 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-24T17:08:19.669 INFO:teuthology.task.internal:Saving configuration 2026-02-24T17:08:19.676 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-24T17:08:19.689 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-24T17:08:19.769 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial066.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial/68438', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 17:04:49.817741', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII5zy2zMNKu4Ndu7nvABenrNhGh2kUk9kkTFhn/8EOCz'} 2026-02-24T17:08:19.769 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-24T17:08:19.781 INFO:teuthology.task.internal:roles: ubuntu@trial066.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-24T17:08:19.781 INFO:teuthology.run_tasks:Running task console_log... 2026-02-24T17:08:19.862 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc7727ae840>, signals=[15]) 2026-02-24T17:08:19.862 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-24T17:08:19.876 INFO:teuthology.task.internal:Opening connections... 2026-02-24T17:08:19.876 DEBUG:teuthology.task.internal:connecting to ubuntu@trial066.front.sepia.ceph.com 2026-02-24T17:08:19.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T17:08:19.942 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-24T17:08:19.954 DEBUG:teuthology.orchestra.run.trial066:> uname -m 2026-02-24T17:08:20.021 INFO:teuthology.orchestra.run.trial066.stdout:x86_64 2026-02-24T17:08:20.021 DEBUG:teuthology.orchestra.run.trial066:> cat /etc/os-release 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:NAME="Ubuntu" 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:VERSION_ID="22.04" 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:VERSION_CODENAME=jammy 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:ID=ubuntu 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:ID_LIKE=debian 2026-02-24T17:08:20.067 INFO:teuthology.orchestra.run.trial066.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-24T17:08:20.068 INFO:teuthology.orchestra.run.trial066.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-24T17:08:20.068 INFO:teuthology.orchestra.run.trial066.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-24T17:08:20.068 INFO:teuthology.orchestra.run.trial066.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-24T17:08:20.068 INFO:teuthology.orchestra.run.trial066.stdout:UBUNTU_CODENAME=jammy 2026-02-24T17:08:20.068 INFO:teuthology.lock.ops:Updating trial066.front.sepia.ceph.com on lock server 2026-02-24T17:08:20.150 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-24T17:08:20.172 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-24T17:08:20.184 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-24T17:08:20.184 DEBUG:teuthology.orchestra.run.trial066:> test '!' -e /home/ubuntu/cephtest 2026-02-24T17:08:20.188 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-24T17:08:20.208 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-24T17:08:20.208 DEBUG:teuthology.orchestra.run.trial066:> test -z $(ls -A /var/lib/ceph) 2026-02-24T17:08:20.236 INFO:teuthology.orchestra.run.trial066.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-24T17:08:20.237 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-24T17:08:20.329 INFO:teuthology.run_tasks:Running task kernel... 2026-02-24T17:08:20.348 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-24T17:08:20.348 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-24T17:08:20.348 DEBUG:teuthology.orchestra.run.trial066:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T17:08:20.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T17:08:20.351 DEBUG:teuthology.orchestra.run.trial066:> uname -r 2026-02-24T17:08:20.395 INFO:teuthology.orchestra.run.trial066.stdout:5.15.0-170-generic 2026-02-24T17:08:20.395 INFO:teuthology.task.kernel:Running kernel on trial066: 5.15.0-170-generic 2026-02-24T17:08:20.395 DEBUG:teuthology.orchestra.run.trial066:> sudo apt-get clean 2026-02-24T17:08:20.453 DEBUG:teuthology.orchestra.run.trial066:> sudo apt-get update 2026-02-24T17:08:20.958 INFO:teuthology.orchestra.run.trial066.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T17:08:21.048 INFO:teuthology.orchestra.run.trial066.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-24T17:08:21.387 INFO:teuthology.orchestra.run.trial066.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-24T17:08:21.537 INFO:teuthology.orchestra.run.trial066.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-24T17:08:21.677 INFO:teuthology.orchestra.run.trial066.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3269 kB] 2026-02-24T17:08:22.080 INFO:teuthology.orchestra.run.trial066.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [497 kB] 2026-02-24T17:08:22.092 INFO:teuthology.orchestra.run.trial066.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.2 kB] 2026-02-24T17:08:22.093 INFO:teuthology.orchestra.run.trial066.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-24T17:08:22.128 INFO:teuthology.orchestra.run.trial066.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-24T17:08:22.134 INFO:teuthology.orchestra.run.trial066.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.4 kB] 2026-02-24T17:08:22.135 INFO:teuthology.orchestra.run.trial066.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5379 kB] 2026-02-24T17:08:22.292 INFO:teuthology.orchestra.run.trial066.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1022 kB] 2026-02-24T17:08:22.317 INFO:teuthology.orchestra.run.trial066.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [3005 kB] 2026-02-24T17:08:22.403 INFO:teuthology.orchestra.run.trial066.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [429 kB] 2026-02-24T17:08:22.414 INFO:teuthology.orchestra.run.trial066.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-24T17:08:22.415 INFO:teuthology.orchestra.run.trial066.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-24T17:08:22.448 INFO:teuthology.orchestra.run.trial066.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [225 kB] 2026-02-24T17:08:22.455 INFO:teuthology.orchestra.run.trial066.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-24T17:08:22.457 INFO:teuthology.orchestra.run.trial066.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5204 kB] 2026-02-24T17:08:22.606 INFO:teuthology.orchestra.run.trial066.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [990 kB] 2026-02-24T17:08:26.049 INFO:teuthology.orchestra.run.trial066.stdout:Fetched 23.1 MB in 3s (7887 kB/s) 2026-02-24T17:08:26.576 INFO:teuthology.orchestra.run.trial066.stdout:Reading package lists... 2026-02-24T17:08:26.589 DEBUG:teuthology.orchestra.run.trial066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T17:08:26.622 INFO:teuthology.orchestra.run.trial066.stdout:Reading package lists... 2026-02-24T17:08:26.728 INFO:teuthology.orchestra.run.trial066.stdout:Building dependency tree... 2026-02-24T17:08:26.729 INFO:teuthology.orchestra.run.trial066.stdout:Reading state information... 2026-02-24T17:08:26.816 INFO:teuthology.orchestra.run.trial066.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T17:08:26.816 INFO:teuthology.orchestra.run.trial066.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T17:08:26.816 INFO:teuthology.orchestra.run.trial066.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-24T17:08:26.816 INFO:teuthology.orchestra.run.trial066.stdout: sg3-utils-udev 2026-02-24T17:08:26.816 INFO:teuthology.orchestra.run.trial066.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T17:08:26.816 INFO:teuthology.orchestra.run.trial066.stdout:The following additional packages will be installed: 2026-02-24T17:08:26.816 INFO:teuthology.orchestra.run.trial066.stdout: linux-generic linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T17:08:26.817 INFO:teuthology.orchestra.run.trial066.stdout: linux-headers-generic linux-image-5.15.0-171-generic 2026-02-24T17:08:26.817 INFO:teuthology.orchestra.run.trial066.stdout: linux-modules-5.15.0-171-generic linux-modules-extra-5.15.0-171-generic 2026-02-24T17:08:26.817 INFO:teuthology.orchestra.run.trial066.stdout:Suggested packages: 2026-02-24T17:08:26.817 INFO:teuthology.orchestra.run.trial066.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-24T17:08:26.829 INFO:teuthology.orchestra.run.trial066.stdout:The following NEW packages will be installed: 2026-02-24T17:08:26.830 INFO:teuthology.orchestra.run.trial066.stdout: linux-headers-5.15.0-171 linux-headers-5.15.0-171-generic 2026-02-24T17:08:26.830 INFO:teuthology.orchestra.run.trial066.stdout: linux-image-5.15.0-171-generic linux-modules-5.15.0-171-generic 2026-02-24T17:08:26.830 INFO:teuthology.orchestra.run.trial066.stdout: linux-modules-extra-5.15.0-171-generic 2026-02-24T17:08:26.830 INFO:teuthology.orchestra.run.trial066.stdout:The following packages will be upgraded: 2026-02-24T17:08:26.830 INFO:teuthology.orchestra.run.trial066.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-24T17:08:27.174 INFO:teuthology.orchestra.run.trial066.stdout:3 upgraded, 5 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T17:08:27.174 INFO:teuthology.orchestra.run.trial066.stdout:Need to get 113 MB of archives. 2026-02-24T17:08:27.174 INFO:teuthology.orchestra.run.trial066.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-24T17:08:27.175 INFO:teuthology.orchestra.run.trial066.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-171-generic amd64 5.15.0-171.181 [22.7 MB] 2026-02-24T17:08:28.270 INFO:teuthology.orchestra.run.trial066.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-171-generic amd64 5.15.0-171.181 [11.6 MB] 2026-02-24T17:08:28.540 INFO:teuthology.orchestra.run.trial066.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-171-generic amd64 5.15.0-171.181 [63.9 MB] 2026-02-24T17:08:30.036 INFO:teuthology.orchestra.run.trial066.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.171.160 [1658 B] 2026-02-24T17:08:30.036 INFO:teuthology.orchestra.run.trial066.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.171.160 [2486 B] 2026-02-24T17:08:30.037 INFO:teuthology.orchestra.run.trial066.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171 all 5.15.0-171.181 [12.4 MB] 2026-02-24T17:08:30.315 INFO:teuthology.orchestra.run.trial066.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-171-generic amd64 5.15.0-171.181 [2849 kB] 2026-02-24T17:08:30.380 INFO:teuthology.orchestra.run.trial066.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.171.160 [2332 B] 2026-02-24T17:08:30.578 INFO:teuthology.orchestra.run.trial066.stdout:Fetched 113 MB in 4s (32.0 MB/s) 2026-02-24T17:08:30.595 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package linux-modules-5.15.0-171-generic. 2026-02-24T17:08:30.769 INFO:teuthology.orchestra.run.trial066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-24T17:08:30.771 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../0-linux-modules-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T17:08:30.773 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:31.104 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package linux-image-5.15.0-171-generic. 2026-02-24T17:08:31.113 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../1-linux-image-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T17:08:31.121 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:31.161 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-171-generic. 2026-02-24T17:08:31.169 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T17:08:31.170 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:32.293 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../3-linux-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T17:08:32.294 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T17:08:32.319 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T17:08:32.320 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-image-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T17:08:32.337 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package linux-headers-5.15.0-171. 2026-02-24T17:08:32.347 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../5-linux-headers-5.15.0-171_5.15.0-171.181_all.deb ... 2026-02-24T17:08:32.347 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T17:08:33.749 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package linux-headers-5.15.0-171-generic. 2026-02-24T17:08:33.764 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../6-linux-headers-5.15.0-171-generic_5.15.0-171.181_amd64.deb ... 2026-02-24T17:08:33.764 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:34.348 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.171.160_amd64.deb ... 2026-02-24T17:08:34.349 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking linux-headers-generic (5.15.0.171.160) over (5.15.0.170.159) ... 2026-02-24T17:08:34.375 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-headers-5.15.0-171 (5.15.0-171.181) ... 2026-02-24T17:08:34.376 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-headers-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:34.378 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-headers-generic (5.15.0.171.160) ... 2026-02-24T17:08:34.379 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:35.147 INFO:teuthology.orchestra.run.trial066.stdout:I: /boot/vmlinuz.old is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-24T17:08:35.148 INFO:teuthology.orchestra.run.trial066.stdout:I: /boot/initrd.img.old is now a symlink to initrd.img-5.15.0-170-generic 2026-02-24T17:08:35.148 INFO:teuthology.orchestra.run.trial066.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-171-generic 2026-02-24T17:08:35.148 INFO:teuthology.orchestra.run.trial066.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-171-generic 2026-02-24T17:08:35.155 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-modules-extra-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:35.944 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-modules-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:36.728 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-image-generic (5.15.0.171.160) ... 2026-02-24T17:08:36.729 INFO:teuthology.orchestra.run.trial066.stdout:Setting up linux-generic (5.15.0.171.160) ... 2026-02-24T17:08:36.730 INFO:teuthology.orchestra.run.trial066.stdout:Processing triggers for linux-image-5.15.0-171-generic (5.15.0-171.181) ... 2026-02-24T17:08:36.735 INFO:teuthology.orchestra.run.trial066.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-24T17:08:36.735 INFO:teuthology.orchestra.run.trial066.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-171-generic 2026-02-24T17:08:43.805 INFO:teuthology.orchestra.run.trial066.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-24T17:08:50.467 INFO:teuthology.orchestra.run.trial066.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-24T17:08:50.467 INFO:teuthology.orchestra.run.trial066.stdout:Sourcing file `/etc/default/grub' 2026-02-24T17:08:50.487 INFO:teuthology.orchestra.run.trial066.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T17:08:50.488 INFO:teuthology.orchestra.run.trial066.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T17:08:50.490 INFO:teuthology.orchestra.run.trial066.stdout:Generating grub configuration file ... 2026-02-24T17:08:50.577 INFO:teuthology.orchestra.run.trial066.stdout:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T17:08:50.584 INFO:teuthology.orchestra.run.trial066.stdout:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T17:08:50.765 INFO:teuthology.orchestra.run.trial066.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T17:08:50.770 INFO:teuthology.orchestra.run.trial066.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T17:08:50.815 INFO:teuthology.orchestra.run.trial066.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T17:08:50.820 INFO:teuthology.orchestra.run.trial066.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T17:08:50.897 INFO:teuthology.orchestra.run.trial066.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T17:08:50.897 INFO:teuthology.orchestra.run.trial066.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T17:08:50.898 INFO:teuthology.orchestra.run.trial066.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T17:08:50.900 INFO:teuthology.orchestra.run.trial066.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T17:08:50.909 INFO:teuthology.orchestra.run.trial066.stdout:done 2026-02-24T17:08:51.314 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout:Pending kernel upgrade! 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout:Running kernel version: 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout: 5.15.0-170-generic 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout:Diagnostics: 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout: The currently running kernel version is not the expected kernel version 5.15.0-171-generic. 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.315 INFO:teuthology.orchestra.run.trial066.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout:The processor microcode seems to be up-to-date. 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout:No services need to be restarted. 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout:No containers need to be restarted. 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout:No user sessions are running outdated binaries. 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:08:51.316 INFO:teuthology.orchestra.run.trial066.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T17:08:52.141 DEBUG:teuthology.orchestra.run.trial066:> dpkg -s linux-image-generic 2026-02-24T17:08:52.151 INFO:teuthology.orchestra.run.trial066.stdout:Package: linux-image-generic 2026-02-24T17:08:52.151 INFO:teuthology.orchestra.run.trial066.stdout:Status: install ok installed 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Priority: optional 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Section: kernel 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Installed-Size: 22 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Architecture: amd64 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Source: linux-meta 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Version: 5.15.0.171.160 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T17:08:52.152 INFO:teuthology.orchestra.run.trial066.stdout:Recommends: thermald 2026-02-24T17:08:52.153 INFO:teuthology.orchestra.run.trial066.stdout:Description: Generic Linux kernel image 2026-02-24T17:08:52.153 INFO:teuthology.orchestra.run.trial066.stdout: This package will always depend on the latest generic kernel image 2026-02-24T17:08:52.153 INFO:teuthology.orchestra.run.trial066.stdout: available. 2026-02-24T17:08:52.153 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-170-generic Expected: 5.15.0-171-generic 2026-02-24T17:08:52.153 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T17:08:52.153 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T17:08:52.153 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-24T17:08:52.154 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial066.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T17:08:52.154 DEBUG:teuthology.orchestra.run.trial066:> sudo apt-get clean 2026-02-24T17:08:52.232 DEBUG:teuthology.orchestra.run.trial066:> sudo apt-get update 2026-02-24T17:08:52.433 INFO:teuthology.orchestra.run.trial066.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T17:08:52.438 INFO:teuthology.orchestra.run.trial066.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T17:08:52.448 INFO:teuthology.orchestra.run.trial066.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T17:08:52.458 INFO:teuthology.orchestra.run.trial066.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T17:08:53.295 INFO:teuthology.orchestra.run.trial066.stdout:Reading package lists... 2026-02-24T17:08:53.310 DEBUG:teuthology.orchestra.run.trial066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-24T17:08:53.344 INFO:teuthology.orchestra.run.trial066.stdout:Reading package lists... 2026-02-24T17:08:53.455 INFO:teuthology.orchestra.run.trial066.stdout:Building dependency tree... 2026-02-24T17:08:53.456 INFO:teuthology.orchestra.run.trial066.stdout:Reading state information... 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout:linux-image-generic is already the newest version (5.15.0.171.160). 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout:linux-image-generic set to manually installed. 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T17:08:53.544 INFO:teuthology.orchestra.run.trial066.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T17:08:53.545 INFO:teuthology.orchestra.run.trial066.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T17:08:53.564 INFO:teuthology.orchestra.run.trial066.stdout:0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T17:08:53.566 DEBUG:teuthology.orchestra.run.trial066:> dpkg -s linux-image-generic 2026-02-24T17:08:53.618 INFO:teuthology.orchestra.run.trial066.stdout:Package: linux-image-generic 2026-02-24T17:08:53.618 INFO:teuthology.orchestra.run.trial066.stdout:Status: install ok installed 2026-02-24T17:08:53.618 INFO:teuthology.orchestra.run.trial066.stdout:Priority: optional 2026-02-24T17:08:53.618 INFO:teuthology.orchestra.run.trial066.stdout:Section: kernel 2026-02-24T17:08:53.618 INFO:teuthology.orchestra.run.trial066.stdout:Installed-Size: 22 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Maintainer: Ubuntu Kernel Team 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Architecture: amd64 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Source: linux-meta 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Version: 5.15.0.171.160 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-171), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Depends: linux-image-5.15.0-171-generic, linux-modules-extra-5.15.0-171-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Recommends: thermald 2026-02-24T17:08:53.619 INFO:teuthology.orchestra.run.trial066.stdout:Description: Generic Linux kernel image 2026-02-24T17:08:53.620 INFO:teuthology.orchestra.run.trial066.stdout: This package will always depend on the latest generic kernel image 2026-02-24T17:08:53.620 INFO:teuthology.orchestra.run.trial066.stdout: available. 2026-02-24T17:08:53.620 DEBUG:teuthology.orchestra.run.trial066:> mktemp 2026-02-24T17:08:53.664 INFO:teuthology.orchestra.run.trial066.stdout:/tmp/tmp.smvoGzJHK4 2026-02-24T17:08:53.665 DEBUG:teuthology.orchestra.run.trial066:> sudo cp /boot/grub/grub.cfg /tmp/tmp.smvoGzJHK4 2026-02-24T17:08:53.717 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod 0666 /tmp/tmp.smvoGzJHK4 2026-02-24T17:08:53.815 DEBUG:teuthology.orchestra.remote:trial066:/tmp/tmp.smvoGzJHK4 is 9KB 2026-02-24T17:08:53.869 DEBUG:teuthology.orchestra.run.trial066:> rm -fr /tmp/tmp.smvoGzJHK4 2026-02-24T17:08:53.873 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-24T17:08:53.924 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:08:53.924 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-24T17:08:53.924 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-24T17:08:53.978 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-171-generic 2026-02-24T17:08:53.979 DEBUG:teuthology.orchestra.run.trial066:> sudo update-grub 2026-02-24T17:08:54.204 INFO:teuthology.orchestra.run.trial066.stderr:Sourcing file `/etc/default/grub' 2026-02-24T17:08:54.222 INFO:teuthology.orchestra.run.trial066.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-24T17:08:54.223 INFO:teuthology.orchestra.run.trial066.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-24T17:08:54.226 INFO:teuthology.orchestra.run.trial066.stderr:Generating grub configuration file ... 2026-02-24T17:08:54.313 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-5.15.0-171-generic 2026-02-24T17:08:54.319 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initrd.img-5.15.0-171-generic 2026-02-24T17:08:54.495 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-24T17:08:54.501 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-24T17:08:54.544 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-24T17:08:54.549 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-24T17:08:54.622 INFO:teuthology.orchestra.run.trial066.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-24T17:08:54.622 INFO:teuthology.orchestra.run.trial066.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-24T17:08:54.622 INFO:teuthology.orchestra.run.trial066.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-24T17:08:54.625 INFO:teuthology.orchestra.run.trial066.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T17:08:54.632 INFO:teuthology.orchestra.run.trial066.stderr:done 2026-02-24T17:08:54.634 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial066 2026-02-24T17:08:54.634 DEBUG:teuthology.orchestra.run.trial066:> sudo bash -c ' 2026-02-24T17:08:54.635 DEBUG:teuthology.orchestra.run.trial066:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T17:08:54.635 DEBUG:teuthology.orchestra.run.trial066:> ' 2026-02-24T17:09:24.637 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-24T17:09:24.638 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-24T17:09:24.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T17:09:38.944 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.66 2026-02-24T17:09:47.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-24T17:09:47.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T17:10:24.003 DEBUG:teuthology.orchestra.run.trial066:> true 2026-02-24T17:10:24.394 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-24T17:10:24.395 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-171-generic"... 2026-02-24T17:10:24.395 DEBUG:teuthology.orchestra.run.trial066:> uname -r 2026-02-24T17:10:24.443 INFO:teuthology.orchestra.run.trial066.stdout:5.15.0-171-generic 2026-02-24T17:10:24.443 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-171-generic vs 5.15.0-171-generic 2026-02-24T17:10:24.444 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T17:10:24.444 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-24T17:10:25.445 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-24T17:10:25.445 DEBUG:teuthology.orchestra.run.trial066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T17:10:25.458 INFO:teuthology.orchestra.run.trial066.stdout:ttyS1 2026-02-24T17:10:25.485 DEBUG:teuthology.parallel:result is None 2026-02-24T17:10:25.485 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-24T17:10:25.497 INFO:teuthology.task.internal:Creating test directory... 2026-02-24T17:10:25.497 DEBUG:teuthology.orchestra.run.trial066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T17:10:25.530 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-24T17:10:25.542 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-24T17:10:25.557 INFO:teuthology.task.internal:Creating archive directory... 2026-02-24T17:10:25.557 DEBUG:teuthology.orchestra.run.trial066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T17:10:25.580 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-24T17:10:25.592 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-24T17:10:25.592 DEBUG:teuthology.orchestra.run.trial066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T17:10:25.627 INFO:teuthology.orchestra.run.trial066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:10:25.633 INFO:teuthology.orchestra.run.trial066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:10:25.635 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-24T17:10:25.646 INFO:teuthology.task.internal:Configuring sudo... 2026-02-24T17:10:25.647 DEBUG:teuthology.orchestra.run.trial066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T17:10:25.686 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-24T17:10:25.703 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-24T17:10:25.703 DEBUG:teuthology.orchestra.run.trial066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T17:10:25.734 DEBUG:teuthology.orchestra.run.trial066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T17:10:25.778 DEBUG:teuthology.orchestra.run.trial066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T17:10:25.822 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:10:25.823 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T17:10:25.875 DEBUG:teuthology.orchestra.run.trial066:> sudo service rsyslog restart 2026-02-24T17:10:25.937 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-24T17:10:25.957 INFO:teuthology.task.internal:Starting timer... 2026-02-24T17:10:25.957 INFO:teuthology.run_tasks:Running task pcp... 2026-02-24T17:10:25.978 INFO:teuthology.run_tasks:Running task selinux... 2026-02-24T17:10:25.994 DEBUG:teuthology.task.selinux:Excluding trial066: OS 'ubuntu' does not support SELinux 2026-02-24T17:10:25.994 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-24T17:10:25.994 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-24T17:10:25.994 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-24T17:10:25.995 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-24T17:10:26.010 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-24T17:10:26.022 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-24T17:10:26.023 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial066.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-24T17:12:51.069 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial066.front.sepia.ceph.com')] 2026-02-24T17:12:51.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-24T17:12:51.073 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T17:12:51.179 DEBUG:teuthology.orchestra.run.trial066:> true 2026-02-24T17:12:51.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-24T17:12:51.427 INFO:teuthology.run_tasks:Running task clock... 2026-02-24T17:12:51.447 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-24T17:12:51.447 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T17:12:51.448 DEBUG:teuthology.orchestra.run.trial066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T17:12:51.492 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-24T17:12:51.492 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Command line: ntpd -gq 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: ---------------------------------------------------- 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: ntp-4 is maintained by Network Time Foundation, 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: corporation. Support and training for ntp-4 are 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: available at https://www.nwtime.org/support 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: ---------------------------------------------------- 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: proto: precision = 0.021 usec (-25) 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: basedate set to 2022-02-04 2026-02-24T17:12:51.493 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: gps base set to 2022-02-06 (week 2196) 2026-02-24T17:12:51.494 INFO:teuthology.orchestra.run.trial066.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T17:12:51.494 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-24T17:12:51.494 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: restrict ::: KOD does nothing without LIMITED. 2026-02-24T17:12:51.495 INFO:teuthology.orchestra.run.trial066.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-24T17:12:51.496 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Listen and drop on 0 v6wildcard [::]:123 2026-02-24T17:12:51.496 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-24T17:12:51.496 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Listen normally on 2 lo 127.0.0.1:123 2026-02-24T17:12:51.496 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Listen normally on 3 enp1s0f0 10.20.193.66:123 2026-02-24T17:12:51.496 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Listen normally on 4 lo [::1]:123 2026-02-24T17:12:51.496 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:5f2c%2]:123 2026-02-24T17:12:51.496 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:51 ntpd[8813]: Listening on routing socket on fd #22 for interface updates 2026-02-24T17:12:52.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:52 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:52.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:52 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:52.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:52 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:52.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:52 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:52.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:52 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:52.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:52 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:54.491 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:54.491 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:54.491 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:54.491 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:54.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:54.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:54.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:54.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:54.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:54.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:54.563 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:54.563 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:54 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:55.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:55.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:55.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:55.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:55.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:55.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:55.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:55.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:55.559 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:55.559 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:55.559 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:55.559 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:55 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:56.491 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:56.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:56.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:56.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:56.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:56.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:56.520 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:56.520 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:56.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:56.563 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:56.563 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:56.563 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:56 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:57.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:57.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:57.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:57.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:57.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:57.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:57.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:57.501 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:57.558 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:57.559 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:57.559 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:57.559 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:57 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:58.491 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:58.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:58.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:58.492 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:58.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:58.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:58.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:58.519 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:58.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-24T17:12:58.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: can't open /var/log/ntpstats/rawstats.20260224: Permission denied 2026-02-24T17:12:58.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-24T17:12:58.562 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: can't open /var/log/ntpstats/peerstats.20260224: Permission denied 2026-02-24T17:12:58.562 INFO:teuthology.orchestra.run.trial066.stdout:24 Feb 17:12:58 ntpd[8813]: ntpd: time set +0.350229 s 2026-02-24T17:12:58.562 INFO:teuthology.orchestra.run.trial066.stdout:ntpd: time set +0.350229s 2026-02-24T17:12:58.563 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-24T17:12:58.563 INFO:teuthology.orchestra.run.trial066.stderr:24 Feb 17:12:58 ntpd[8813]: can't open /var/log/ntpstats/loopstats.20260224: Permission denied 2026-02-24T17:12:58.635 INFO:teuthology.orchestra.run.trial066.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T17:12:58.636 INFO:teuthology.orchestra.run.trial066.stdout:============================================================================== 2026-02-24T17:12:58.636 INFO:teuthology.orchestra.run.trial066.stdout: ntp.swyn.net .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T17:12:58.636 INFO:teuthology.orchestra.run.trial066.stdout: clock.xmission. .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T17:12:58.636 INFO:teuthology.orchestra.run.trial066.stdout: time.cloudflare .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T17:12:58.636 INFO:teuthology.orchestra.run.trial066.stdout: t1.time.bf1.yah .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-24T17:12:58.636 INFO:teuthology.run_tasks:Running task install... 2026-02-24T17:12:58.649 DEBUG:teuthology.task.install:project ceph 2026-02-24T17:12:58.649 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '618c318a7a0b287ab5aeff207b7bf1d73ef45044'}} 2026-02-24T17:12:58.649 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '618c318a7a0b287ab5aeff207b7bf1d73ef45044'} 2026-02-24T17:12:58.650 INFO:teuthology.task.install:Using flavor: default 2026-02-24T17:12:58.654 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-24T17:12:58.654 INFO:teuthology.task.install:extra packages: [] 2026-02-24T17:12:58.655 DEBUG:teuthology.orchestra.run.trial066:> sudo apt-key list | grep Ceph 2026-02-24T17:12:58.735 INFO:teuthology.orchestra.run.trial066.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-24T17:12:58.764 INFO:teuthology.orchestra.run.trial066.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-24T17:12:58.764 INFO:teuthology.orchestra.run.trial066.stdout:uid [ unknown] Ceph.com (release key) 2026-02-24T17:12:58.765 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-24T17:12:58.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T17:12:58.933 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default/ 2026-02-24T17:12:58.933 INFO:teuthology.task.install.deb:Package version is 20.3.0-5507-g618c318a-1jammy 2026-02-24T17:12:59.088 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:12:59.088 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-24T17:12:59.099 DEBUG:teuthology.orchestra.run.trial066:> sudo apt-get update 2026-02-24T17:12:59.249 INFO:teuthology.orchestra.run.trial066.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy InRelease 2026-02-24T17:12:59.260 INFO:teuthology.orchestra.run.trial066.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-24T17:12:59.272 INFO:teuthology.orchestra.run.trial066.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-24T17:12:59.284 INFO:teuthology.orchestra.run.trial066.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-24T17:12:59.548 INFO:teuthology.orchestra.run.trial066.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-24T17:12:59.616 INFO:teuthology.orchestra.run.trial066.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-24T17:12:59.688 INFO:teuthology.orchestra.run.trial066.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-24T17:12:59.760 INFO:teuthology.orchestra.run.trial066.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-24T17:12:59.903 INFO:teuthology.orchestra.run.trial066.stdout:Fetched 26.9 kB in 1s (41.4 kB/s) 2026-02-24T17:13:00.420 INFO:teuthology.orchestra.run.trial066.stdout:Reading package lists... 2026-02-24T17:13:00.434 DEBUG:teuthology.orchestra.run.trial066:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5507-g618c318a-1jammy cephadm=20.3.0-5507-g618c318a-1jammy ceph-mds=20.3.0-5507-g618c318a-1jammy ceph-mgr=20.3.0-5507-g618c318a-1jammy ceph-osd=20.3.0-5507-g618c318a-1jammy ceph-osd-classic=20.3.0-5507-g618c318a-1jammy ceph-common=20.3.0-5507-g618c318a-1jammy ceph-fuse=20.3.0-5507-g618c318a-1jammy ceph-test=20.3.0-5507-g618c318a-1jammy ceph-volume=20.3.0-5507-g618c318a-1jammy radosgw=20.3.0-5507-g618c318a-1jammy python3-rados=20.3.0-5507-g618c318a-1jammy python3-rgw=20.3.0-5507-g618c318a-1jammy python3-cephfs=20.3.0-5507-g618c318a-1jammy python3-rbd=20.3.0-5507-g618c318a-1jammy libcephfs2=20.3.0-5507-g618c318a-1jammy libcephfs-dev=20.3.0-5507-g618c318a-1jammy librados2=20.3.0-5507-g618c318a-1jammy librbd1=20.3.0-5507-g618c318a-1jammy rbd-fuse=20.3.0-5507-g618c318a-1jammy 2026-02-24T17:13:00.467 INFO:teuthology.orchestra.run.trial066.stdout:Reading package lists... 2026-02-24T17:13:00.577 INFO:teuthology.orchestra.run.trial066.stdout:Building dependency tree... 2026-02-24T17:13:00.578 INFO:teuthology.orchestra.run.trial066.stdout:Reading state information... 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout:The following packages were automatically installed and are no longer required: 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: linux-headers-5.15.0-164 linux-headers-5.15.0-164-generic 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: linux-image-5.15.0-164-generic linux-modules-5.15.0-164-generic 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: linux-modules-extra-5.15.0-164-generic sg3-utils sg3-utils-udev 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout:The following additional packages will be installed: 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-24T17:13:00.668 INFO:teuthology.orchestra.run.trial066.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout:Suggested packages: 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout:Recommended packages: 2026-02-24T17:13:00.669 INFO:teuthology.orchestra.run.trial066.stdout: btrfs-tools 2026-02-24T17:13:00.703 INFO:teuthology.orchestra.run.trial066.stdout:The following NEW packages will be installed: 2026-02-24T17:13:00.703 INFO:teuthology.orchestra.run.trial066.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-24T17:13:00.703 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-24T17:13:00.703 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-24T17:13:00.703 INFO:teuthology.orchestra.run.trial066.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-24T17:13:00.703 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-24T17:13:00.703 INFO:teuthology.orchestra.run.trial066.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-24T17:13:00.704 INFO:teuthology.orchestra.run.trial066.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-24T17:13:00.705 INFO:teuthology.orchestra.run.trial066.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-24T17:13:00.705 INFO:teuthology.orchestra.run.trial066.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-24T17:13:00.705 INFO:teuthology.orchestra.run.trial066.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-24T17:13:00.705 INFO:teuthology.orchestra.run.trial066.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-24T17:13:00.705 INFO:teuthology.orchestra.run.trial066.stdout: socat xmlstarlet 2026-02-24T17:13:00.785 INFO:teuthology.orchestra.run.trial066.stdout:0 upgraded, 87 newly installed, 0 to remove and 14 not upgraded. 2026-02-24T17:13:00.785 INFO:teuthology.orchestra.run.trial066.stdout:Need to get 693 MB of archives. 2026-02-24T17:13:00.785 INFO:teuthology.orchestra.run.trial066.stdout:After this operation, 2583 MB of additional disk space will be used. 2026-02-24T17:13:00.785 INFO:teuthology.orchestra.run.trial066.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5507-g618c318a-1jammy [3796 kB] 2026-02-24T17:13:00.893 INFO:teuthology.orchestra.run.trial066.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5507-g618c318a-1jammy [2910 kB] 2026-02-24T17:13:00.932 INFO:teuthology.orchestra.run.trial066.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5507-g618c318a-1jammy [883 kB] 2026-02-24T17:13:00.947 INFO:teuthology.orchestra.run.trial066.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5507-g618c318a-1jammy [359 kB] 2026-02-24T17:13:00.953 INFO:teuthology.orchestra.run.trial066.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5507-g618c318a-1jammy [32.8 kB] 2026-02-24T17:13:00.953 INFO:teuthology.orchestra.run.trial066.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5507-g618c318a-1jammy [225 kB] 2026-02-24T17:13:00.957 INFO:teuthology.orchestra.run.trial066.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5507-g618c318a-1jammy [88.9 kB] 2026-02-24T17:13:00.958 INFO:teuthology.orchestra.run.trial066.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5507-g618c318a-1jammy [338 kB] 2026-02-24T17:13:00.964 INFO:teuthology.orchestra.run.trial066.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5507-g618c318a-1jammy [9130 kB] 2026-02-24T17:13:01.102 INFO:teuthology.orchestra.run.trial066.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-24T17:13:01.114 INFO:teuthology.orchestra.run.trial066.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5507-g618c318a-1jammy [112 kB] 2026-02-24T17:13:01.115 INFO:teuthology.orchestra.run.trial066.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5507-g618c318a-1jammy [261 kB] 2026-02-24T17:13:01.119 INFO:teuthology.orchestra.run.trial066.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5507-g618c318a-1jammy [31.6 MB] 2026-02-24T17:13:01.489 INFO:teuthology.orchestra.run.trial066.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-24T17:13:01.502 INFO:teuthology.orchestra.run.trial066.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-24T17:13:01.581 INFO:teuthology.orchestra.run.trial066.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-24T17:13:01.645 INFO:teuthology.orchestra.run.trial066.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5507-g618c318a-1jammy [4984 kB] 2026-02-24T17:13:01.720 INFO:teuthology.orchestra.run.trial066.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5507-g618c318a-1jammy [295 kB] 2026-02-24T17:13:01.724 INFO:teuthology.orchestra.run.trial066.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5507-g618c318a-1jammy [127 kB] 2026-02-24T17:13:01.726 INFO:teuthology.orchestra.run.trial066.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5507-g618c318a-1jammy [934 kB] 2026-02-24T17:13:01.741 INFO:teuthology.orchestra.run.trial066.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5507-g618c318a-1jammy [7550 kB] 2026-02-24T17:13:01.812 INFO:teuthology.orchestra.run.trial066.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-24T17:13:01.824 INFO:teuthology.orchestra.run.trial066.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-24T17:13:01.855 INFO:teuthology.orchestra.run.trial066.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-24T17:13:01.864 INFO:teuthology.orchestra.run.trial066.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-24T17:13:01.866 INFO:teuthology.orchestra.run.trial066.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5507-g618c318a-1jammy [468 MB] 2026-02-24T17:13:01.866 INFO:teuthology.orchestra.run.trial066.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-24T17:13:01.867 INFO:teuthology.orchestra.run.trial066.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-24T17:13:01.868 INFO:teuthology.orchestra.run.trial066.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-24T17:13:01.887 INFO:teuthology.orchestra.run.trial066.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-24T17:13:01.887 INFO:teuthology.orchestra.run.trial066.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-24T17:13:01.894 INFO:teuthology.orchestra.run.trial066.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-24T17:13:01.971 INFO:teuthology.orchestra.run.trial066.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-24T17:13:01.972 INFO:teuthology.orchestra.run.trial066.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-24T17:13:01.972 INFO:teuthology.orchestra.run.trial066.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-24T17:13:01.973 INFO:teuthology.orchestra.run.trial066.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-24T17:13:01.973 INFO:teuthology.orchestra.run.trial066.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-24T17:13:01.973 INFO:teuthology.orchestra.run.trial066.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-24T17:13:01.973 INFO:teuthology.orchestra.run.trial066.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-24T17:13:01.978 INFO:teuthology.orchestra.run.trial066.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-24T17:13:01.978 INFO:teuthology.orchestra.run.trial066.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-24T17:13:02.058 INFO:teuthology.orchestra.run.trial066.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-24T17:13:02.061 INFO:teuthology.orchestra.run.trial066.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-24T17:13:02.061 INFO:teuthology.orchestra.run.trial066.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-24T17:13:02.062 INFO:teuthology.orchestra.run.trial066.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-24T17:13:02.090 INFO:teuthology.orchestra.run.trial066.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-24T17:13:02.092 INFO:teuthology.orchestra.run.trial066.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-24T17:13:02.093 INFO:teuthology.orchestra.run.trial066.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-24T17:13:02.160 INFO:teuthology.orchestra.run.trial066.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-24T17:13:02.160 INFO:teuthology.orchestra.run.trial066.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-24T17:13:02.161 INFO:teuthology.orchestra.run.trial066.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-24T17:13:02.161 INFO:teuthology.orchestra.run.trial066.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-24T17:13:02.161 INFO:teuthology.orchestra.run.trial066.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-24T17:13:02.162 INFO:teuthology.orchestra.run.trial066.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-24T17:13:02.229 INFO:teuthology.orchestra.run.trial066.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-24T17:13:02.230 INFO:teuthology.orchestra.run.trial066.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-24T17:13:02.232 INFO:teuthology.orchestra.run.trial066.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-24T17:13:02.312 INFO:teuthology.orchestra.run.trial066.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-24T17:13:02.315 INFO:teuthology.orchestra.run.trial066.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-24T17:13:02.317 INFO:teuthology.orchestra.run.trial066.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-24T17:13:02.322 INFO:teuthology.orchestra.run.trial066.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-24T17:13:02.324 INFO:teuthology.orchestra.run.trial066.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-24T17:13:02.325 INFO:teuthology.orchestra.run.trial066.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-24T17:13:02.325 INFO:teuthology.orchestra.run.trial066.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-24T17:13:02.395 INFO:teuthology.orchestra.run.trial066.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-24T17:13:02.395 INFO:teuthology.orchestra.run.trial066.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-24T17:13:02.402 INFO:teuthology.orchestra.run.trial066.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-24T17:13:02.479 INFO:teuthology.orchestra.run.trial066.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-24T17:13:02.484 INFO:teuthology.orchestra.run.trial066.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-24T17:13:02.484 INFO:teuthology.orchestra.run.trial066.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-24T17:13:02.485 INFO:teuthology.orchestra.run.trial066.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-24T17:13:13.933 INFO:teuthology.orchestra.run.trial066.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5507-g618c318a-1jammy [180 kB] 2026-02-24T17:13:13.935 INFO:teuthology.orchestra.run.trial066.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5507-g618c318a-1jammy [14.1 kB] 2026-02-24T17:13:13.935 INFO:teuthology.orchestra.run.trial066.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5507-g618c318a-1jammy [1030 kB] 2026-02-24T17:13:13.940 INFO:teuthology.orchestra.run.trial066.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5507-g618c318a-1jammy [2437 kB] 2026-02-24T17:13:13.952 INFO:teuthology.orchestra.run.trial066.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5507-g618c318a-1jammy [1072 kB] 2026-02-24T17:13:13.979 INFO:teuthology.orchestra.run.trial066.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5507-g618c318a-1jammy [190 kB] 2026-02-24T17:13:13.982 INFO:teuthology.orchestra.run.trial066.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5507-g618c318a-1jammy [49.0 MB] 2026-02-24T17:13:15.568 INFO:teuthology.orchestra.run.trial066.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5507-g618c318a-1jammy [8626 kB] 2026-02-24T17:13:15.839 INFO:teuthology.orchestra.run.trial066.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5507-g618c318a-1jammy [14.2 kB] 2026-02-24T17:13:15.839 INFO:teuthology.orchestra.run.trial066.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5507-g618c318a-1jammy [65.5 MB] 2026-02-24T17:13:17.787 INFO:teuthology.orchestra.run.trial066.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5507-g618c318a-1jammy [136 kB] 2026-02-24T17:13:17.787 INFO:teuthology.orchestra.run.trial066.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5507-g618c318a-1jammy [47.3 kB] 2026-02-24T17:13:17.788 INFO:teuthology.orchestra.run.trial066.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5507-g618c318a-1jammy [33.1 kB] 2026-02-24T17:13:17.788 INFO:teuthology.orchestra.run.trial066.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5507-g618c318a-1jammy [44.2 kB] 2026-02-24T17:13:17.788 INFO:teuthology.orchestra.run.trial066.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5507-g618c318a-1jammy [18.4 MB] 2026-02-24T17:13:18.321 INFO:teuthology.orchestra.run.trial066.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri8-testing-2026-02-23-2128/618c318a7a0b287ab5aeff207b7bf1d73ef45044/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5507-g618c318a-1jammy [93.7 kB] 2026-02-24T17:13:18.596 INFO:teuthology.orchestra.run.trial066.stdout:Fetched 693 MB in 18s (39.4 MB/s) 2026-02-24T17:13:18.612 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-24T17:13:18.781 INFO:teuthology.orchestra.run.trial066.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 202926 files and directories currently installed.) 2026-02-24T17:13:18.783 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-24T17:13:18.784 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T17:13:18.805 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-24T17:13:18.820 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-24T17:13:18.821 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T17:13:18.838 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-24T17:13:18.852 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-24T17:13:18.852 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T17:13:18.870 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-24T17:13:18.884 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T17:13:18.890 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T17:13:18.921 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-24T17:13:18.931 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T17:13:18.932 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T17:13:18.949 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-24T17:13:18.959 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-24T17:13:18.960 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T17:13:18.980 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-24T17:13:18.990 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-24T17:13:18.991 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T17:13:19.008 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package librados2. 2026-02-24T17:13:19.018 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../07-librados2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.018 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking librados2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.061 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libnbd0. 2026-02-24T17:13:19.071 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-24T17:13:19.072 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-24T17:13:19.087 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package librbd1. 2026-02-24T17:13:19.097 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../09-librbd1_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.097 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking librbd1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.135 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libcephfs2. 2026-02-24T17:13:19.145 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.146 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libcephfs2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.165 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-rados. 2026-02-24T17:13:19.175 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.176 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-rados (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.194 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-24T17:13:19.205 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:19.205 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-ceph-argparse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.221 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-cephfs. 2026-02-24T17:13:19.230 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.231 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-cephfs (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.247 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-24T17:13:19.258 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:19.259 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.277 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-24T17:13:19.287 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-24T17:13:19.287 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T17:13:19.303 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-prettytable. 2026-02-24T17:13:19.313 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-24T17:13:19.314 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-24T17:13:19.329 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-rbd. 2026-02-24T17:13:19.338 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.339 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-rbd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.356 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-24T17:13:19.367 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-24T17:13:19.367 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T17:13:19.386 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package librgw2. 2026-02-24T17:13:19.396 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../19-librgw2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.396 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking librgw2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.493 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-rgw. 2026-02-24T17:13:19.504 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.504 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-rgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.521 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-24T17:13:19.530 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-24T17:13:19.531 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T17:13:19.546 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libradosstriper1. 2026-02-24T17:13:19.557 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.557 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libradosstriper1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.575 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-common. 2026-02-24T17:13:19.585 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.585 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.850 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-base. 2026-02-24T17:13:19.861 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:19.865 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-base (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:19.927 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-24T17:13:19.938 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-24T17:13:19.938 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-24T17:13:19.954 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-cheroot. 2026-02-24T17:13:19.964 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-24T17:13:19.965 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T17:13:19.982 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-24T17:13:19.993 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-24T17:13:19.993 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-24T17:13:20.008 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-24T17:13:20.018 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-24T17:13:20.019 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-24T17:13:20.033 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-24T17:13:20.044 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-24T17:13:20.044 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-24T17:13:20.060 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-tempora. 2026-02-24T17:13:20.070 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-24T17:13:20.070 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-24T17:13:20.086 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-portend. 2026-02-24T17:13:20.096 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-24T17:13:20.097 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-24T17:13:20.112 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-24T17:13:20.122 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-24T17:13:20.122 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-24T17:13:20.137 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-24T17:13:20.147 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-24T17:13:20.148 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-24T17:13:20.174 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-natsort. 2026-02-24T17:13:20.184 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-24T17:13:20.184 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-24T17:13:20.203 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-24T17:13:20.213 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:20.214 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:20.245 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-24T17:13:20.255 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:20.255 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:20.271 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mgr. 2026-02-24T17:13:20.281 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:20.281 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mgr (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:20.303 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mon. 2026-02-24T17:13:20.313 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:20.314 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:20.381 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-24T17:13:20.391 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-24T17:13:20.392 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T17:13:20.407 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-24T17:13:20.417 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:20.418 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-osd-classic (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:24.271 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-osd. 2026-02-24T17:13:24.286 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:24.286 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-osd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:24.306 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph. 2026-02-24T17:13:24.321 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../42-ceph_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:24.322 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:24.338 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-fuse. 2026-02-24T17:13:24.353 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:24.354 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:24.377 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mds. 2026-02-24T17:13:24.388 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:24.389 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mds (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:24.421 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package cephadm. 2026-02-24T17:13:24.431 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../45-cephadm_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:24.432 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:24.450 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-24T17:13:24.460 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T17:13:24.460 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T17:13:24.481 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-24T17:13:24.492 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:24.492 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:24.514 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-24T17:13:24.525 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-24T17:13:24.525 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-24T17:13:24.541 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-routes. 2026-02-24T17:13:24.551 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-24T17:13:24.551 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T17:13:24.571 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-24T17:13:24.581 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:24.582 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:25.124 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-24T17:13:25.136 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-24T17:13:25.136 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T17:13:25.178 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-joblib. 2026-02-24T17:13:25.188 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-24T17:13:25.189 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T17:13:25.216 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-24T17:13:25.226 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-24T17:13:25.226 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-24T17:13:25.241 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-sklearn. 2026-02-24T17:13:25.252 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-24T17:13:25.252 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T17:13:25.333 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-24T17:13:25.345 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:25.345 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:25.505 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-cachetools. 2026-02-24T17:13:25.516 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-24T17:13:25.516 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-24T17:13:25.531 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-rsa. 2026-02-24T17:13:25.541 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-24T17:13:25.541 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-24T17:13:25.561 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-google-auth. 2026-02-24T17:13:25.571 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-24T17:13:25.572 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-24T17:13:25.589 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-24T17:13:25.600 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-24T17:13:25.600 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T17:13:25.617 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-websocket. 2026-02-24T17:13:25.627 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-24T17:13:25.628 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-24T17:13:25.646 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-24T17:13:25.656 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-24T17:13:25.657 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T17:13:25.753 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-24T17:13:25.764 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:25.765 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:25.779 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-24T17:13:25.789 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-24T17:13:25.790 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T17:13:25.807 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-24T17:13:25.817 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T17:13:25.818 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T17:13:25.833 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package jq. 2026-02-24T17:13:25.843 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-24T17:13:25.843 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-24T17:13:25.859 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package socat. 2026-02-24T17:13:25.869 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-24T17:13:25.870 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-24T17:13:25.891 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package xmlstarlet. 2026-02-24T17:13:25.901 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-24T17:13:25.901 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-24T17:13:25.938 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-test. 2026-02-24T17:13:25.949 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:25.949 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-test (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:26.573 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package ceph-volume. 2026-02-24T17:13:26.584 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5507-g618c318a-1jammy_all.deb ... 2026-02-24T17:13:26.585 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking ceph-volume (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:26.609 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-24T17:13:26.619 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:26.619 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libcephfs-daemon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:26.635 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-24T17:13:26.646 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:26.647 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libcephfs-proxy2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:26.661 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-24T17:13:26.672 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:26.672 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking libcephfs-dev (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:26.689 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package nvme-cli. 2026-02-24T17:13:26.699 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-24T17:13:26.699 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T17:13:26.731 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-24T17:13:26.742 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-24T17:13:26.742 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T17:13:26.773 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-24T17:13:26.783 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-24T17:13:26.783 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-24T17:13:26.799 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-pluggy. 2026-02-24T17:13:26.809 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-24T17:13:26.809 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-24T17:13:26.825 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-psutil. 2026-02-24T17:13:26.836 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-24T17:13:26.836 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-24T17:13:26.855 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-py. 2026-02-24T17:13:26.866 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-24T17:13:26.866 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-24T17:13:26.887 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-pygments. 2026-02-24T17:13:26.897 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-24T17:13:26.898 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T17:13:26.939 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-toml. 2026-02-24T17:13:26.950 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-24T17:13:26.950 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-24T17:13:26.966 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-pytest. 2026-02-24T17:13:26.976 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-24T17:13:26.976 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T17:13:26.999 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-simplejson. 2026-02-24T17:13:27.009 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-24T17:13:27.009 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-24T17:13:27.035 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package python3-webob. 2026-02-24T17:13:27.045 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-24T17:13:27.046 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T17:13:27.064 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-24T17:13:27.074 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-24T17:13:27.075 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-24T17:13:27.144 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package radosgw. 2026-02-24T17:13:27.155 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../85-radosgw_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:27.155 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking radosgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:27.310 INFO:teuthology.orchestra.run.trial066.stdout:Selecting previously unselected package rbd-fuse. 2026-02-24T17:13:27.321 INFO:teuthology.orchestra.run.trial066.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5507-g618c318a-1jammy_amd64.deb ... 2026-02-24T17:13:27.321 INFO:teuthology.orchestra.run.trial066.stdout:Unpacking rbd-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:27.350 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-24T17:13:27.416 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-24T17:13:27.417 INFO:teuthology.orchestra.run.trial066.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-24T17:13:27.488 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T17:13:27.776 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-24T17:13:28.169 INFO:teuthology.orchestra.run.trial066.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-24T17:13:28.180 INFO:teuthology.orchestra.run.trial066.stdout:Setting up cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:28.224 INFO:teuthology.orchestra.run.trial066.stdout:Adding system user cephadm....done 2026-02-24T17:13:28.230 INFO:teuthology.orchestra.run.trial066.stdout:usermod: no changes 2026-02-24T17:13:28.236 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-24T17:13:28.308 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-24T17:13:28.309 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-24T17:13:28.381 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-24T17:13:28.455 INFO:teuthology.orchestra.run.trial066.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-24T17:13:28.456 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-24T17:13:28.550 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-24T17:13:28.671 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-24T17:13:28.745 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-24T17:13:28.818 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-ceph-argparse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:28.883 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-24T17:13:28.884 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-24T17:13:28.885 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-24T17:13:28.885 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-24T17:13:28.886 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-24T17:13:29.008 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-24T17:13:29.086 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libcephfs-proxy2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:29.087 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-24T17:13:29.162 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-24T17:13:29.252 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-24T17:13:29.500 INFO:teuthology.orchestra.run.trial066.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-24T17:13:29.501 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-24T17:13:29.599 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-24T17:13:29.724 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-24T17:13:29.805 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-24T17:13:29.877 INFO:teuthology.orchestra.run.trial066.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-24T17:13:29.878 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:29.967 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-24T17:13:30.457 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T17:13:30.461 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-24T17:13:30.537 INFO:teuthology.orchestra.run.trial066.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-24T17:13:30.537 INFO:teuthology.orchestra.run.trial066.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-24T17:13:30.538 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-24T17:13:30.615 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-24T17:13:30.684 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T17:13:30.685 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-24T17:13:30.762 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-24T17:13:30.839 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-24T17:13:30.918 INFO:teuthology.orchestra.run.trial066.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-24T17:13:30.919 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-24T17:13:30.998 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-24T17:13:30.999 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-24T17:13:31.076 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-24T17:13:31.159 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-24T17:13:31.233 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-24T17:13:31.234 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-24T17:13:31.367 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-24T17:13:31.440 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-24T17:13:31.441 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-24T17:13:31.522 INFO:teuthology.orchestra.run.trial066.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-24T17:13:31.523 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-24T17:13:31.654 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-24T17:13:31.655 INFO:teuthology.orchestra.run.trial066.stdout:Setting up librados2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:31.656 INFO:teuthology.orchestra.run.trial066.stdout:Setting up librgw2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:31.657 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:31.657 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-24T17:13:32.168 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libcephfs2 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.169 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libradosstriper1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.170 INFO:teuthology.orchestra.run.trial066.stdout:Setting up librbd1 (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.170 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mgr-modules-core (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.171 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.247 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T17:13:32.247 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-24T17:13:32.662 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libcephfs-dev (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.663 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-rados (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.664 INFO:teuthology.orchestra.run.trial066.stdout:Setting up libcephfs-daemon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.665 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-rbd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.666 INFO:teuthology.orchestra.run.trial066.stdout:Setting up rbd-fuse (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.667 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-rgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.667 INFO:teuthology.orchestra.run.trial066.stdout:Setting up python3-cephfs (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.668 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-common (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:32.696 INFO:teuthology.orchestra.run.trial066.stdout:Adding group ceph....done 2026-02-24T17:13:32.732 INFO:teuthology.orchestra.run.trial066.stdout:Adding system user ceph....done 2026-02-24T17:13:32.742 INFO:teuthology.orchestra.run.trial066.stdout:Setting system user ceph properties....done 2026-02-24T17:13:32.748 INFO:teuthology.orchestra.run.trial066.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-24T17:13:32.832 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-24T17:13:33.108 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-24T17:13:33.505 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-test (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:33.506 INFO:teuthology.orchestra.run.trial066.stdout:Setting up radosgw (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:33.774 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T17:13:33.774 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-24T17:13:34.167 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-base (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:34.264 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-24T17:13:34.655 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mds (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:34.714 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T17:13:34.714 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-24T17:13:35.117 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mgr (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:35.196 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T17:13:35.196 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-24T17:13:35.602 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:35.602 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-osd-classic (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:35.619 INFO:teuthology.orchestra.run.trial066.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-24T17:13:35.629 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:35.640 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mon (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:35.712 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T17:13:35.713 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-24T17:13:36.101 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mgr-cephadm (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:36.111 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-mgr-dashboard (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:36.121 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-osd (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:36.196 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T17:13:36.196 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-24T17:13:36.607 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:36.607 INFO:teuthology.orchestra.run.trial066.stdout:Setting up ceph-volume (20.3.0-5507-g618c318a-1jammy) ... 2026-02-24T17:13:36.716 INFO:teuthology.orchestra.run.trial066.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-24T17:13:36.801 INFO:teuthology.orchestra.run.trial066.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-24T17:13:37.460 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:13:37.460 INFO:teuthology.orchestra.run.trial066.stdout:Running kernel seems to be up-to-date. 2026-02-24T17:13:37.460 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout:The processor microcode seems to be up-to-date. 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout:No services need to be restarted. 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout:No containers need to be restarted. 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout:No user sessions are running outdated binaries. 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-24T17:13:37.461 INFO:teuthology.orchestra.run.trial066.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-24T17:13:38.260 INFO:teuthology.orchestra.run.trial066.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-24T17:13:38.265 DEBUG:teuthology.parallel:result is None 2026-02-24T17:13:38.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T17:13:38.408 DEBUG:teuthology.orchestra.run.trial066:> dpkg-query -W -f '${Version}' ceph 2026-02-24T17:13:38.417 INFO:teuthology.orchestra.run.trial066.stdout:20.3.0-5507-g618c318a-1jammy 2026-02-24T17:13:38.418 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5507-g618c318a-1jammy 2026-02-24T17:13:38.418 INFO:teuthology.task.install:The correct ceph version 20.3.0-5507-g618c318a-1jammy is installed. 2026-02-24T17:13:38.420 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-24T17:13:38.420 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:13:38.420 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T17:13:38.468 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-24T17:13:38.468 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:13:38.468 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T17:13:38.520 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T17:13:38.570 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-24T17:13:38.570 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:13:38.571 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T17:13:38.619 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T17:13:38.667 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-24T17:13:38.668 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:13:38.668 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T17:13:38.721 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T17:13:38.771 INFO:teuthology.run_tasks:Running task workunit... 2026-02-24T17:13:38.811 INFO:tasks.workunit:Pulling workunits from ref 618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T17:13:38.811 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-24T17:13:38.811 INFO:tasks.workunit:timeout=3h 2026-02-24T17:13:38.811 INFO:tasks.workunit:cleanup=True 2026-02-24T17:13:38.812 DEBUG:teuthology.orchestra.run.trial066:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-24T17:13:38.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T17:13:38.816 INFO:teuthology.orchestra.run.trial066.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-24T17:13:38.816 DEBUG:teuthology.orchestra.run.trial066:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T17:13:38.863 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-24T17:13:38.864 DEBUG:teuthology.orchestra.run.trial066:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-24T17:13:38.911 DEBUG:teuthology.orchestra.run.trial066:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 618c318a7a0b287ab5aeff207b7bf1d73ef45044 2026-02-24T17:13:38.963 INFO:tasks.workunit.client.0.trial066.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-24T17:14:00.708 INFO:tasks.workunit.client.0.trial066.stderr:Note: switching to '618c318a7a0b287ab5aeff207b7bf1d73ef45044'. 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr: 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr:state without impacting any branches by switching back to a branch. 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr: 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr: 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr: git switch -c 2026-02-24T17:14:00.709 INFO:tasks.workunit.client.0.trial066.stderr: 2026-02-24T17:14:00.710 INFO:tasks.workunit.client.0.trial066.stderr:Or undo this operation with: 2026-02-24T17:14:00.710 INFO:tasks.workunit.client.0.trial066.stderr: 2026-02-24T17:14:00.710 INFO:tasks.workunit.client.0.trial066.stderr: git switch - 2026-02-24T17:14:00.710 INFO:tasks.workunit.client.0.trial066.stderr: 2026-02-24T17:14:00.710 INFO:tasks.workunit.client.0.trial066.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-24T17:14:00.710 INFO:tasks.workunit.client.0.trial066.stderr: 2026-02-24T17:14:00.710 INFO:tasks.workunit.client.0.trial066.stderr:HEAD is now at 618c318a7a0 Merge branch 'stretched-cluster' of https://github.com/leonidc/ceph into wip-yuri8-testing-2026-02-23-2128 2026-02-24T17:14:00.718 DEBUG:teuthology.orchestra.run.trial066:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-24T17:14:00.765 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-24T17:14:00.765 DEBUG:teuthology.orchestra.run.trial066:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-24T17:14:00.811 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-24T17:14:00.811 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-24T17:14:00.811 DEBUG:teuthology.orchestra.run.trial066:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-24T17:14:00.860 INFO:tasks.workunit.client.0.trial066.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-24T17:14:00.860 INFO:tasks.workunit.client.0.trial066.stderr:++ TIMEOUT=300 2026-02-24T17:14:00.860 INFO:tasks.workunit.client.0.trial066.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-24T17:14:00.860 INFO:tasks.workunit.client.0.trial066.stderr:++ MAX_TIMEOUT=15 2026-02-24T17:14:00.860 INFO:tasks.workunit.client.0.trial066.stderr:++ PG_NUM=4 2026-02-24T17:14:00.861 INFO:tasks.workunit.client.0.trial066.stderr:++ TMPDIR=/tmp 2026-02-24T17:14:00.861 INFO:tasks.workunit.client.0.trial066.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-24T17:14:00.861 INFO:tasks.workunit.client.0.trial066.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-24T17:14:00.861 INFO:tasks.workunit.client.0.trial066.stderr:+++ uname 2026-02-24T17:14:00.861 INFO:tasks.workunit.client.0.trial066.stderr:++ '[' Linux = FreeBSD ']' 2026-02-24T17:14:00.861 INFO:tasks.workunit.client.0.trial066.stderr:++ SED=sed 2026-02-24T17:14:00.861 INFO:tasks.workunit.client.0.trial066.stderr:++ AWK=awk 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:+++ stty -a 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:+++ head -1 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:++ termwidth= 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:++ DIFFCOLOPTS='-y ' 2026-02-24T17:14:00.863 INFO:tasks.workunit.client.0.trial066.stderr:++ KERNCORE=kernel.core_pattern 2026-02-24T17:14:00.864 INFO:tasks.workunit.client.0.trial066.stderr:++ EXTRA_OPTS= 2026-02-24T17:14:00.865 INFO:tasks.workunit.client.0.trial066.stderr:++ test '' = TESTS 2026-02-24T17:14:00.865 INFO:tasks.workunit.client.0.trial066.stderr:++ uname -m 2026-02-24T17:14:00.868 INFO:tasks.workunit.client.0.trial066.stderr:+ arch=x86_64 2026-02-24T17:14:00.868 INFO:tasks.workunit.client.0.trial066.stderr:+ case $arch in 2026-02-24T17:14:00.868 INFO:tasks.workunit.client.0.trial066.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-24T17:14:00.868 INFO:tasks.workunit.client.0.trial066.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:+ main test-erasure-code-plugins 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:+ shift 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:+ shopt -s -o xtrace 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T17:14:00.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/test-erasure-code-plugins 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-24T17:14:00.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none ' 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_blaum_roth_warning 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:TEST_ec_profile_warning 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:TEST_preload_no_warning 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:TEST_preload_no_warning_default 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:TEST_preload_warning' 2026-02-24T17:14:00.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_blaum_roth_warning td/test-erasure-code-plugins 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:119: TEST_ec_profile_blaum_roth_warning: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:121: TEST_ec_profile_blaum_roth_warning: setup td/test-erasure-code-plugins 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:00.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:00.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:00.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:00.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:00.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:00.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:00.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:00.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:00.877 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:00.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:00.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:00.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:00.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:00.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:00.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:00.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:00.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:00.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:00.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:00.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:00.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:00.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:00.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:14:00.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:14:00.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:00.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:00.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:122: TEST_ec_profile_blaum_roth_warning: run_mon td/test-erasure-code-plugins a 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:14:00.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:00.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:14:00.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:14:00.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:14:00.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:00.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:00.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:14:00.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:14:00.958 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:00.959 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:01.013 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:01.014 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:01.014 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:01.014 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:01.014 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:01.014 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:01.014 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:14:01.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:123: TEST_ec_profile_blaum_roth_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:14:01.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:01.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:14:01.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:14:01.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:14:01.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:14:01.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:14:01.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:14:01.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:01.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:01.175 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:01.175 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:01.175 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:01.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:01.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:01.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:01.198 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: seq 0 2 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 0 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:01.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:01.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ac627579-f74a-492d-b2ae-68cfe2b274bd 2026-02-24T17:14:01.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ac627579-f74a-492d-b2ae-68cfe2b274bd' 2026-02-24T17:14:01.202 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 ac627579-f74a-492d-b2ae-68cfe2b274bd 2026-02-24T17:14:01.202 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:01.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZ3J1pDQl4DBAAp0b3JtFYujPk2+gkuJRMZA== 2026-02-24T17:14:01.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZ3J1pDQl4DBAAp0b3JtFYujPk2+gkuJRMZA=="}' 2026-02-24T17:14:01.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ac627579-f74a-492d-b2ae-68cfe2b274bd -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:01.326 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:01.334 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:01.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZ3J1pDQl4DBAAp0b3JtFYujPk2+gkuJRMZA== --osd-uuid ac627579-f74a-492d-b2ae-68cfe2b274bd 2026-02-24T17:14:01.350 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:01.342+0000 7fbd29aa9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:01.352 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:01.346+0000 7fbd29aa9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:01.353 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:01.346+0000 7fbd29aa9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:01.353 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:01.346+0000 7fbd29aa9a40 -1 bdev(0x56154021e400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:01.353 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:01.346+0000 7fbd29aa9a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:14:02.086 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:14:02.086 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:02.087 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:14:02.087 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:14:02.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:02.205 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:14:02.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:14:02.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:14:02.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:02.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:02.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:02.219 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:02.210+0000 7fce6affaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:02.221 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:02.214+0000 7fce6affaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:02.222 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:02.214+0000 7fce6affaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:02.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:14:02.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:02.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:14:02.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:02.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:02.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:02.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:02.321 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:02.322 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:02.322 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:02.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:02.721 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:02.714+0000 7fce6affaa40 -1 Falling back to public interface 2026-02-24T17:14:03.426 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:03.418+0000 7fce6affaa40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:14:03.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:03.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:03.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:03.428 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:03.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:03.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:03.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:04.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:04.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:04.535 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:04.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:14:04.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:04.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:04.755 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2691775845,v1:127.0.0.1:6801/2691775845] [v2:127.0.0.1:6802/2691775845,v1:127.0.0.1:6803/2691775845] exists,up ac627579-f74a-492d-b2ae-68cfe2b274bd 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 1 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:14:04.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:04.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:04.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:04.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:04.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:04.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:04.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:14:04.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-24T17:14:04.762 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:04.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e3161730-6b80-4782-be13-bf57489be3f4 2026-02-24T17:14:04.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e3161730-6b80-4782-be13-bf57489be3f4' 2026-02-24T17:14:04.762 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 e3161730-6b80-4782-be13-bf57489be3f4 2026-02-24T17:14:04.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:04.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBc3J1pn4arLRAA9aPvyRguGqPUskwAcorWOw== 2026-02-24T17:14:04.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBc3J1pn4arLRAA9aPvyRguGqPUskwAcorWOw=="}' 2026-02-24T17:14:04.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e3161730-6b80-4782-be13-bf57489be3f4 -i td/test-erasure-code-plugins/1/new.json 2026-02-24T17:14:04.986 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:04.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-24T17:14:04.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBc3J1pn4arLRAA9aPvyRguGqPUskwAcorWOw== --osd-uuid e3161730-6b80-4782-be13-bf57489be3f4 2026-02-24T17:14:05.021 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.014+0000 7fa6db68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:05.022 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.014+0000 7fa6db68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:05.023 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.014+0000 7fa6db68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:05.024 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.014+0000 7fa6db68ea40 -1 bdev(0x5647e99ee400 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:05.024 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.014+0000 7fa6db68ea40 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-24T17:14:05.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-24T17:14:05.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:05.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:14:05.665 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:14:05.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:05.969 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:14:05.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:14:05.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:14:05.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:05.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:05.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:05.991 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.982+0000 7fcfd33a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:05.993 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.986+0000 7fcfd33a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:05.994 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:05.986+0000 7fcfd33a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:06.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:14:06.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:06.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:14:06.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:06.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:06.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:06.181 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:06.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:06.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:06.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:14:06.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:06.401 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:06.394+0000 7fcfd33a8a40 -1 Falling back to public interface 2026-02-24T17:14:06.993 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:06.986+0000 7fcfd33a8a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:14:07.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:07.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:07.401 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:07.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:07.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:07.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:14:07.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:08.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:08.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:08.611 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:08.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:14:08.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:08.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:14:08.828 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2499230715,v1:127.0.0.1:6811/2499230715] [v2:127.0.0.1:6812/2499230715,v1:127.0.0.1:6813/2499230715] exists,up e3161730-6b80-4782-be13-bf57489be3f4 2026-02-24T17:14:08.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:08.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:08.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:08.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:124: TEST_ec_profile_blaum_roth_warning: for id in $(seq 0 2) 2026-02-24T17:14:08.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:125: TEST_ec_profile_blaum_roth_warning: run_osd td/test-erasure-code-plugins 2 2026-02-24T17:14:08.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-24T17:14:08.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:08.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:14:08.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-24T17:14:08.835 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:08.835 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d10d3582-e861-4974-b864-a5a653219f16 2026-02-24T17:14:08.835 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d10d3582-e861-4974-b864-a5a653219f16' 2026-02-24T17:14:08.836 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 d10d3582-e861-4974-b864-a5a653219f16 2026-02-24T17:14:08.836 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:08.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBg3J1ph9UXMhAAwh9lzFdx9cbUtVsygiZKBw== 2026-02-24T17:14:08.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBg3J1ph9UXMhAAwh9lzFdx9cbUtVsygiZKBw=="}' 2026-02-24T17:14:08.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d10d3582-e861-4974-b864-a5a653219f16 -i td/test-erasure-code-plugins/2/new.json 2026-02-24T17:14:09.057 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:09.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-24T17:14:09.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBg3J1ph9UXMhAAwh9lzFdx9cbUtVsygiZKBw== --osd-uuid d10d3582-e861-4974-b864-a5a653219f16 2026-02-24T17:14:09.082 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:09.074+0000 7f8bce3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:09.083 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:09.074+0000 7f8bce3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:09.084 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:09.078+0000 7f8bce3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:09.084 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:09.078+0000 7f8bce3d6a40 -1 bdev(0x5584cc524400 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:09.085 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:09.078+0000 7f8bce3d6a40 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-24T17:14:09.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-24T17:14:09.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:09.716 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:14:09.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:14:09.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:10.012 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:14:10.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:14:10.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:14:10.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:10.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:10.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:10.028 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:10.022+0000 7fee8eaeca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:10.030 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:10.022+0000 7fee8eaeca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:10.031 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:10.022+0000 7fee8eaeca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:10.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:14:10.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:10.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:14:10.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:10.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:10.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:10.224 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:10.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:10.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:10.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:14:10.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:10.509 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:10.502+0000 7fee8eaeca40 -1 Falling back to public interface 2026-02-24T17:14:11.144 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:11.138+0000 7fee8eaeca40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:14:11.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:11.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:11.442 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:11.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:11.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:11.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:14:11.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:12.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:12.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:12.659 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:12.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:14:12.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:12.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:14:12.871 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3808042224,v1:127.0.0.1:6819/3808042224] [v2:127.0.0.1:6820/3808042224,v1:127.0.0.1:6821/3808042224] exists,up d10d3582-e861-4974-b864-a5a653219f16 2026-02-24T17:14:12.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:12.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:12.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:12.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:127: TEST_ec_profile_blaum_roth_warning: create_rbd_pool 2026-02-24T17:14:12.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-24T17:14:13.079 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' does not exist 2026-02-24T17:14:13.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-24T17:14:13.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:14:13.318 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:14:13.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:14:14.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-24T17:14:14.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:128: TEST_ec_profile_blaum_roth_warning: wait_for_clean 2026-02-24T17:14:14.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:14:14.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:14:14.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:14:14.610 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:14:14.610 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:14:14.610 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:14:14.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:14:14.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:14:14.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:14:14.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:14:14.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:14:14.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:14:14.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:14:14.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:14:14.710 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:14:14.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:14:14.916 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:14:14.916 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:14:14.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:14:14.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:14:14.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:14:14.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836484 2026-02-24T17:14:14.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836484 2026-02-24T17:14:14.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484' 2026-02-24T17:14:14.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:14:14.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:14:15.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705668 2026-02-24T17:14:15.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705668 2026-02-24T17:14:15.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705668' 2026-02-24T17:14:15.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:14:15.073 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:14:15.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-24T17:14:15.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-24T17:14:15.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705668 2-55834574852' 2026-02-24T17:14:15.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:14:15.150 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836484 2026-02-24T17:14:15.150 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:14:15.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:14:15.152 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836484 2026-02-24T17:14:15.152 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:14:15.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836484 2026-02-24T17:14:15.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-24T17:14:15.153 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 21474836484 2026-02-24T17:14:15.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:14:15.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-24T17:14:15.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:14:16.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:14:16.366 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:14:16.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-24T17:14:16.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:14:16.584 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705668 2026-02-24T17:14:16.584 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:14:16.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:14:16.585 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705668 2026-02-24T17:14:16.586 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:14:16.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705668 2026-02-24T17:14:16.587 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 38654705668 2026-02-24T17:14:16.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-24T17:14:16.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:14:16.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-24T17:14:16.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:14:16.799 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574852 2026-02-24T17:14:16.799 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:14:16.800 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:14:16.801 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574852 2026-02-24T17:14:16.801 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:14:16.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-24T17:14:16.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-24T17:14:16.802 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 55834574852 2026-02-24T17:14:16.802 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:14:17.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-24T17:14:17.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:14:17.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:14:17.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:14:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:14:17.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-24T17:14:17.532 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:14:17.532 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:14:17.532 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:14:17.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-24T17:14:17.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:14:17.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:14:17.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:130: TEST_ec_profile_blaum_roth_warning: echo 'Starting test for blaum-roth profile health warning' 2026-02-24T17:14:17.819 INFO:tasks.workunit.client.0.trial066.stdout:Starting test for blaum-roth profile health warning 2026-02-24T17:14:17.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:133: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set prof- plugin=jerasure k=3 m=1 technique=blaum_roth w=7 --yes-i-really-mean-it --force 2026-02-24T17:14:18.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:135: TEST_ec_profile_blaum_roth_warning: ceph osd erasure-code-profile set test crush-device-class= crush-failure-domain=osd crush-num-failure-domains=0 crush-osds-per-failure-domain=0 crush-root=default k=4 l=3 m=2 name=lrcprofile plugin=lrc 2026-02-24T17:14:18.442 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: get_asok_path mon.a 2026-02-24T17:14:18.442 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:18.443 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:18.443 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:18.443 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:18.443 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:18.443 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:18.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: CEPH_ARGS= 2026-02-24T17:14:18.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:136: TEST_ec_profile_blaum_roth_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:18.493 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:18.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:137: TEST_ec_profile_blaum_roth_warning: sleep 10 2026-02-24T17:14:28.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:138: TEST_ec_profile_blaum_roth_warning: grep -F '1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:20.946+0000 7f46aa9a9640 0 log_channel(cluster) log [WRN] : Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:20.946+0000 7f46a81a4640 10 mon.a@0(leader).log v84 logging 2026-02-24T17:14:20.951780+0000 mon.a (mon.0) 150 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:20.998+0000 7f46a599f640 7 mon.a@0(leader).log v85 update_from_paxos applying incremental log 85 2026-02-24T17:14:20.951780+0000 mon.a (mon.0) 150 : cluster [WRN] Health check failed: 1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption (BLAUM_ROTH_W_IS_NOT_PRIME) 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stdout: "message": "1 or more EC profiles have a w value such that w+1 is not prime. This can result in data corruption", 2026-02-24T17:14:28.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:139: TEST_ec_profile_blaum_roth_warning: grep -F 'w+1=8 for the EC profile prof- is not prime and could lead to data corruption' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:28.506 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.506 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.506 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.506 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.506 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stdout: "message": "w+1=8 for the EC profile prof- is not prime and could lead to data corruption" 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:141: TEST_ec_profile_blaum_roth_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:28.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:28.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:28.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:28.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:28.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:28.617 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:28.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:28.617 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:28.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:28.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:28.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:28.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:28.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:28.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:28.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:28.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:28.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:28.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:28.633 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:28.633 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.633 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:28.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:28.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:28.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:142: TEST_ec_profile_blaum_roth_warning: return 0 2026-02-24T17:14:28.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-24T17:14:28.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-24T17:14:28.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:28.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:28.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:28.638 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:28.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:28.639 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:28.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:28.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:28.640 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:28.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:28.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:28.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:28.642 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:28.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:28.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:28.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:28.645 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:28.645 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.645 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:28.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:28.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:28.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:14:28.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:14:28.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:14:28.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:14:28.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:14:28.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:14:28.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.675 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:14:28.675 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:28.676 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:28.676 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:28.676 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.676 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.676 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:28.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:14:28.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:14:28.704 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:14:28.704 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:28.704 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:28.704 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:14:28.705 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:14:28.705 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:28.706 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:28.706 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:28.706 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:28.706 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.706 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.706 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:28.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:28.707 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:28.761 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:28.762 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.762 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.762 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:28.762 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:28.762 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:14:28.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:14:28.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:14:28.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:14:28.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:14:28.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:14:28.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:14:28.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:14:28.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:28.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:28.923 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:28.923 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.923 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:28.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:28.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:28.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:14:28.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:14:28.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:14:28.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:28.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=62e19698-2521-4daa-a33e-255ae591f3d8 2026-02-24T17:14:28.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 62e19698-2521-4daa-a33e-255ae591f3d8' 2026-02-24T17:14:28.946 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 62e19698-2521-4daa-a33e-255ae591f3d8 2026-02-24T17:14:28.946 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:28.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB03J1p1gPTOBAAJ7vvu72cvmKhOTHz19K6ZQ== 2026-02-24T17:14:28.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB03J1p1gPTOBAAJ7vvu72cvmKhOTHz19K6ZQ=="}' 2026-02-24T17:14:28.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 62e19698-2521-4daa-a33e-255ae591f3d8 -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:29.070 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:29.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:29.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB03J1p1gPTOBAAJ7vvu72cvmKhOTHz19K6ZQ== --osd-uuid 62e19698-2521-4daa-a33e-255ae591f3d8 2026-02-24T17:14:29.094 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.086+0000 7f985c56da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:29.096 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.090+0000 7f985c56da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:29.097 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.090+0000 7f985c56da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:29.097 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.090+0000 7f985c56da40 -1 bdev(0x55936a810400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:29.098 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.090+0000 7f985c56da40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:14:29.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:14:29.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:29.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:14:29.809 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:14:29.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:29.927 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:14:29.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:14:29.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:14:29.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:29.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:29.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:29.951 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.941+0000 7fa4358efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:29.953 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.945+0000 7fa4358efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:29.954 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:29.945+0000 7fa4358efa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:30.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:14:30.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:30.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:14:30.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:30.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:30.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:30.035 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:30.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:30.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:30.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:30.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:30.526 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:30.517+0000 7fa4358efa40 -1 Falling back to public interface 2026-02-24T17:14:31.141 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:31.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:31.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:31.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:31.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:31.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:31.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:31.506 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:31.497+0000 7fa4358efa40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:14:32.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:32.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:32.254 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:32.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:14:32.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:32.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:32.462 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/154533236,v1:127.0.0.1:6801/154533236] [v2:127.0.0.1:6802/154533236,v1:127.0.0.1:6803/154533236] exists,up 62e19698-2521-4daa-a33e-255ae591f3d8 2026-02-24T17:14:32.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:32.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:32.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:32.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-24T17:14:32.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-24T17:14:32.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-24T17:14:32.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:32.468 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:32.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:32.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:14:32.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-24T17:14:32.470 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:32.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6715c2bd-02f1-4bc3-9cfc-3f62cbcdb400 2026-02-24T17:14:32.471 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 6715c2bd-02f1-4bc3-9cfc-3f62cbcdb400 2026-02-24T17:14:32.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6715c2bd-02f1-4bc3-9cfc-3f62cbcdb400' 2026-02-24T17:14:32.471 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:32.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB43J1pGmdLHBAAqKN0S7Xi+U8FxgZjLu1VCQ== 2026-02-24T17:14:32.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB43J1pGmdLHBAAqKN0S7Xi+U8FxgZjLu1VCQ=="}' 2026-02-24T17:14:32.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6715c2bd-02f1-4bc3-9cfc-3f62cbcdb400 -i td/test-erasure-code-plugins/1/new.json 2026-02-24T17:14:32.695 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:32.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-24T17:14:32.704 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB43J1pGmdLHBAAqKN0S7Xi+U8FxgZjLu1VCQ== --osd-uuid 6715c2bd-02f1-4bc3-9cfc-3f62cbcdb400 2026-02-24T17:14:32.728 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:32.721+0000 7ff77db7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:32.730 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:32.721+0000 7ff77db7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:32.731 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:32.721+0000 7ff77db7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:32.731 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:32.721+0000 7ff77db7fa40 -1 bdev(0x55c74086e400 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:32.732 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:32.725+0000 7ff77db7fa40 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-24T17:14:33.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-24T17:14:33.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:33.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:14:33.369 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:14:33.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:33.652 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:14:33.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:14:33.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:14:33.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:33.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:33.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:33.674 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:33.665+0000 7f8e51d7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:33.676 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:33.669+0000 7f8e51d7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:33.677 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:33.669+0000 7f8e51d7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:33.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:14:34.058 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:34.049+0000 7f8e51d7fa40 -1 Falling back to public interface 2026-02-24T17:14:34.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:34.665 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:34.657+0000 7f8e51d7fa40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:14:35.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:35.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:35.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:35.078 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:35.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:35.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:14:35.288 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1855471231,v1:127.0.0.1:6811/1855471231] [v2:127.0.0.1:6812/1855471231,v1:127.0.0.1:6813/1855471231] exists,up 6715c2bd-02f1-4bc3-9cfc-3f62cbcdb400 2026-02-24T17:14:35.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:35.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:35.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:35.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:35.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:35.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:14:35.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-24T17:14:35.293 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:35.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f061afb1-9a2e-471e-aca5-327daae84910 2026-02-24T17:14:35.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f061afb1-9a2e-471e-aca5-327daae84910' 2026-02-24T17:14:35.294 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 f061afb1-9a2e-471e-aca5-327daae84910 2026-02-24T17:14:35.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:35.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB73J1pqIDRERAACqEb++AJhblM+0Rod/Mt+Q== 2026-02-24T17:14:35.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB73J1pqIDRERAACqEb++AJhblM+0Rod/Mt+Q=="}' 2026-02-24T17:14:35.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f061afb1-9a2e-471e-aca5-327daae84910 -i td/test-erasure-code-plugins/2/new.json 2026-02-24T17:14:35.510 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:35.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-24T17:14:35.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB73J1pqIDRERAACqEb++AJhblM+0Rod/Mt+Q== --osd-uuid f061afb1-9a2e-471e-aca5-327daae84910 2026-02-24T17:14:35.536 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:35.529+0000 7f82f280fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:35.537 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:35.529+0000 7f82f280fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:35.538 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:35.529+0000 7f82f280fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:35.539 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:35.529+0000 7f82f280fa40 -1 bdev(0x5650a90f4400 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:35.539 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:35.529+0000 7f82f280fa40 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-24T17:14:36.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-24T17:14:36.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:36.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:14:36.269 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:14:36.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:36.554 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:14:36.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:14:36.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:14:36.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:36.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:36.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:36.577 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:36.569+0000 7fcad3e64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:36.579 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:36.569+0000 7fcad3e64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:36.580 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:36.573+0000 7fcad3e64a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:36.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:14:36.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:36.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:14:36.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:36.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:36.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:36.770 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:36.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:36.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:36.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:14:36.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:36.989 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:36.981+0000 7fcad3e64a40 -1 Falling back to public interface 2026-02-24T17:14:37.588 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:37.577+0000 7fcad3e64a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:14:37.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:37.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:37.978 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:37.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:37.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:37.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:14:38.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:39.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:39.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:39.186 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:39.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:14:39.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:39.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:14:39.404 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/381097811,v1:127.0.0.1:6819/381097811] [v2:127.0.0.1:6820/381097811,v1:127.0.0.1:6821/381097811] exists,up f061afb1-9a2e-471e-aca5-327daae84910 2026-02-24T17:14:39.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:39.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:39.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:39.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-24T17:14:39.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-24T17:14:39.603 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' does not exist 2026-02-24T17:14:39.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-24T17:14:39.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:14:39.843 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:14:39.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:14:40.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-24T17:14:41.137 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-24T17:14:41.137 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:14:41.137 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:14:41.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:14:41.138 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:14:41.138 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:14:41.139 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:14:41.139 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:14:41.139 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:14:41.139 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:14:41.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:14:41.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:14:41.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:14:41.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:14:41.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:14:41.233 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:14:41.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:14:41.447 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:14:41.447 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:14:41.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:14:41.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:14:41.448 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:14:41.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836484 2026-02-24T17:14:41.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836484 2026-02-24T17:14:41.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484' 2026-02-24T17:14:41.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:14:41.525 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:14:41.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705669 2026-02-24T17:14:41.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705669 2026-02-24T17:14:41.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705669' 2026-02-24T17:14:41.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:14:41.604 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:14:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574851 2026-02-24T17:14:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574851 2026-02-24T17:14:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836484 1-38654705669 2-55834574851' 2026-02-24T17:14:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:14:41.680 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836484 2026-02-24T17:14:41.680 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:14:41.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:14:41.682 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836484 2026-02-24T17:14:41.682 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:14:41.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836484 2026-02-24T17:14:41.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-24T17:14:41.683 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 21474836484 2026-02-24T17:14:41.684 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:14:41.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-24T17:14:41.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:14:42.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:14:42.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:14:43.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-24T17:14:43.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:14:44.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:14:44.119 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:14:44.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836485 -lt 21474836484 2026-02-24T17:14:44.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:14:44.329 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705669 2026-02-24T17:14:44.329 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:14:44.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:14:44.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705669 2026-02-24T17:14:44.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:14:44.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705669 2026-02-24T17:14:44.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-24T17:14:44.332 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 38654705669 2026-02-24T17:14:44.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:14:44.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-24T17:14:44.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:14:44.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574851 2026-02-24T17:14:44.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:14:44.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:14:44.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574851 2026-02-24T17:14:44.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:14:44.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574851 2026-02-24T17:14:44.544 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 55834574851 2026-02-24T17:14:44.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-24T17:14:44.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:14:44.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-24T17:14:44.750 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:14:44.750 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:14:44.750 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:14:45.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-24T17:14:45.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:14:45.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:14:45.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:14:45.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:14:45.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:14:45.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:14:45.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:14:45.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-24T17:14:45.226 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:14:45.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:14:45.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:14:45.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-24T17:14:45.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:14:45.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:14:45.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-24T17:14:45.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:45.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-24T17:14:45.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:45.837 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:45.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:45.845 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:45.657+0000 7fcb06251640 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:45.846 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:45.769+0000 7fcb06251640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:45.846 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:45.769+0000 7fcb06251640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:45.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-24T17:14:45.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-24T17:14:46.164 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-24T17:14:46.164 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:46.164 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:46.164 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:46.164 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:46.164 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:46.164 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:46.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-24T17:14:46.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:46.215 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:46.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:46.224 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:45.997+0000 7fcb06251640 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:46.224 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.145+0000 7fcb06251640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:46.224 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.145+0000 7fcb06251640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:46.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-24T17:14:46.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-24T17:14:46.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-24T17:14:46.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:46.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:46.503 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:46.503 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:46.503 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:46.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:46.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-24T17:14:46.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:46.553 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:46.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:46.560 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.373+0000 7fcb06251640 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:46.560 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.481+0000 7fcb06251640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:46.560 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.481+0000 7fcb06251640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-24T17:14:46.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-24T17:14:46.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-24T17:14:46.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-24T17:14:46.838 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:46.838 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:46.838 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:46.838 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:46.838 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:46.838 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:46.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-24T17:14:46.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:46.887 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:46.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:46.895 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.705+0000 7fcb06251640 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-24T17:14:46.895 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.817+0000 7fcb06251640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-24T17:14:46.895 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:46.817+0000 7fcb06251640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-24T17:14:46.895 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-24T17:14:46.896 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-24T17:14:47.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-24T17:14:47.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:47.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:47.174 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:47.174 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.174 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:47.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-24T17:14:47.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:47.224 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:47.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:47.232 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:47.041+0000 7fcb06251640 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-24T17:14:47.232 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:47.153+0000 7fcb06251640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-24T17:14:47.232 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:47.153+0000 7fcb06251640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-24T17:14:47.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-24T17:14:47.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-24T17:14:47.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-24T17:14:47.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:47.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:47.505 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:47.505 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.505 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:47.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-24T17:14:47.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:47.554 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:47.560 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:47.563 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:47.377+0000 7fcb06251640 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-24T17:14:47.563 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:47.481+0000 7fcb06251640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-24T17:14:47.563 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:14:47.481+0000 7fcb06251640 0 mon.a@0(leader).osd e26 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-24T17:14:47.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:14:47.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:47.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:47.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:47.564 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:47.565 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:47.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:47.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:47.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:47.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:47.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:47.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:47.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:47.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:47.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:47.676 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:47.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:47.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:47.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:47.677 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:47.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:47.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:47.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:47.688 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:47.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.689 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:47.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:47.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:47.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-24T17:14:47.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-24T17:14:47.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:47.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:47.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:47.693 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:47.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:47.694 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:47.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:47.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:47.695 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:47.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:47.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:47.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:47.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:47.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:47.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:47.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:47.700 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:47.700 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.700 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:47.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:47.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:47.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:14:47.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:14:47.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:14:47.704 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:14:47.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:14:47.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:14:47.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-24T17:14:47.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:14:47.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:47.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:47.727 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:47.727 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.727 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.728 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:47.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-24T17:14:47.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:14:47.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:14:47.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:47.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:47.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:14:47.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:14:47.758 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:47.758 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:47.758 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:47.758 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:47.758 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.758 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.758 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:47.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:47.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:14:47.815 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:14:47.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:47.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:47.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:14:47.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:14:47.816 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:47.816 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:47.816 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:47.817 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:47.817 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.817 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.817 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:47.818 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:47.818 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:14:47.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:14:47.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:47.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:14:47.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:14:47.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:14:47.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:14:47.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:14:47.983 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:14:47.983 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:47.983 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:47.983 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:47.983 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:47.984 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:47.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:47.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:47.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:48.004 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-24T17:14:48.004 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:48.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:48.005 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:48.005 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:48.005 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:48.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:48.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:14:48.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:48.055 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:48.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:48.064 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-24T17:14:48.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:14:48.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:48.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ade23d1e-2ac1-4c72-b388-1f55bf518842 2026-02-24T17:14:48.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ade23d1e-2ac1-4c72-b388-1f55bf518842' 2026-02-24T17:14:48.067 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 ade23d1e-2ac1-4c72-b388-1f55bf518842 2026-02-24T17:14:48.067 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:48.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCI3J1pcL1qBBAALDftiA9BK/BCYlg8fZCCwA== 2026-02-24T17:14:48.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCI3J1pcL1qBBAALDftiA9BK/BCYlg8fZCCwA=="}' 2026-02-24T17:14:48.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ade23d1e-2ac1-4c72-b388-1f55bf518842 -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:48.187 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:48.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:48.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQCI3J1pcL1qBBAALDftiA9BK/BCYlg8fZCCwA== --osd-uuid ade23d1e-2ac1-4c72-b388-1f55bf518842 2026-02-24T17:14:48.210 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:48.201+0000 7f9ab1888a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:48.212 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:48.201+0000 7f9ab1888a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:48.213 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:48.205+0000 7f9ab1888a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:48.213 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:48.205+0000 7f9ab1888a40 -1 bdev(0x560983354400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:48.213 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:48.205+0000 7f9ab1888a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:14:48.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:14:48.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:48.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:14:48.977 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:14:48.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:49.100 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:14:49.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:14:49.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-24T17:14:49.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:49.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:49.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:49.114 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:49.105+0000 7fa327097a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:49.116 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:49.105+0000 7fa327097a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:49.117 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:49.109+0000 7fa327097a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:49.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:14:49.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:49.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:14:49.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:49.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:49.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:49.229 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:49.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:49.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:49.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:49.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:49.562 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:49.553+0000 7fa327097a40 -1 Falling back to public interface 2026-02-24T17:14:50.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:50.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:50.340 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:50.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:50.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:50.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:50.501 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:50.493+0000 7fa327097a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:14:50.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:51.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:51.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:51.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:14:51.559 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:51.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:51.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:51.775 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1383297006,v1:127.0.0.1:6801/1383297006] [v2:127.0.0.1:6802/1383297006,v1:127.0.0.1:6803/1383297006] exists,up ade23d1e-2ac1-4c72-b388-1f55bf518842 2026-02-24T17:14:51.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:51.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:51.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:14:51.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:14:51.834 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:51.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:51.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:14:51.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:14:51.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:51.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:51.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:51.843 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:51.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:51.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:51.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:51.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:51.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:51.951 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:51.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:51.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:51.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:51.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:51.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:51.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:51.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:51.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:51.956 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:51.956 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:51.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:51.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:51.961 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:51.961 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:51.961 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:51.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:51.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:51.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:51.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-24T17:14:51.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:51.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:51.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:51.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:51.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:51.966 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:51.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:51.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:51.968 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:51.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:51.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:51.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:51.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:51.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:51.972 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:51.972 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:51.973 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:51.974 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:51.974 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:51.974 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:51.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:51.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:51.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:14:51.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:14:51.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:51.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:51.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:14:51.980 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:14:51.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:14:51.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:14:51.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:14:51.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-24T17:14:51.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:51.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:14:51.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:14:51.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:14:51.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:14:51.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-24T17:14:52.000 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:14:52.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:52.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:52.001 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:52.001 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:52.001 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:52.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:52.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-24T17:14:52.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:14:52.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:14:52.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:52.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:52.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:14:52.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:14:52.030 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:52.030 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:52.030 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:52.031 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:52.031 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:52.031 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:52.031 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:52.031 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:52.031 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:14:52.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:14:52.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:52.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:52.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:14:52.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:14:52.088 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:52.088 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:52.088 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:52.088 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:52.088 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:52.088 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:52.089 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:52.089 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:52.089 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:14:52.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:14:52.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:52.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:14:52.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:14:52.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:14:52.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:14:52.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:14:52.259 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:14:52.259 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:52.260 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:52.260 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:52.260 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:52.260 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:52.260 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:52.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:52.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:52.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-24T17:14:52.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:52.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:52.280 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:52.280 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:52.280 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:52.281 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:52.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:14:52.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:52.330 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:52.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-24T17:14:52.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:14:52.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:52.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:14:52.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:52.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9a616bf4-3080-4b86-b3a7-5a1f37387379 2026-02-24T17:14:52.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9a616bf4-3080-4b86-b3a7-5a1f37387379' 2026-02-24T17:14:52.343 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 9a616bf4-3080-4b86-b3a7-5a1f37387379 2026-02-24T17:14:52.343 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:52.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCM3J1p/M6cFBAA/XdUto+P6MrDoOFku9A50A== 2026-02-24T17:14:52.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCM3J1p/M6cFBAA/XdUto+P6MrDoOFku9A50A=="}' 2026-02-24T17:14:52.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9a616bf4-3080-4b86-b3a7-5a1f37387379 -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:52.460 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:52.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:52.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQCM3J1p/M6cFBAA/XdUto+P6MrDoOFku9A50A== --osd-uuid 9a616bf4-3080-4b86-b3a7-5a1f37387379 2026-02-24T17:14:52.484 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:52.477+0000 7f7f2efe7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:52.486 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:52.477+0000 7f7f2efe7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:52.487 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:52.477+0000 7f7f2efe7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:52.487 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:52.477+0000 7f7f2efe7a40 -1 bdev(0x5555970ba400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:52.487 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:52.477+0000 7f7f2efe7a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:14:53.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:14:53.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:53.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:14:53.370 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:14:53.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:53.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:14:53.487 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:14:53.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-24T17:14:53.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:53.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:53.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:53.512 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:53.505+0000 7f4f1f41ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:53.514 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:53.505+0000 7f4f1f41ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:53.515 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:53.505+0000 7f4f1f41ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:53.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:53.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:53.711 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:54.098 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:54.089+0000 7f4f1f41ba40 -1 Falling back to public interface 2026-02-24T17:14:54.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:54.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:54.713 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:54.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:54.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:54.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:54.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:54.998 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:54.989+0000 7f4f1f41ba40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:14:55.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:55.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:55.928 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:14:55.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:14:55.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:55.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:56.141 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3365902949,v1:127.0.0.1:6801/3365902949] [v2:127.0.0.1:6802/3365902949,v1:127.0.0.1:6803/3365902949] exists,up 9a616bf4-3080-4b86-b3a7-5a1f37387379 2026-02-24T17:14:56.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:14:56.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:14:56.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:14:56.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:14:56.207 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:56.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:14:56.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:56.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:56.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:56.325 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:56.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:56.326 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:56.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:56.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:56.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:56.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:56.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:56.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:56.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:56.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:56.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:56.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:56.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:56.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:56.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:56.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:56.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-24T17:14:56.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-24T17:14:56.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:14:56.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:14:56.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:14:56.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:14:56.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:14:56.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:14:56.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:14:56.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:14:56.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:14:56.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:14:56.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:56.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:14:56.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:14:56.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:14:56.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:14:56.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:14:56.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:14:56.348 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:14:56.349 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.349 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:14:56.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:14:56.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:14:56.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:14:56.351 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:14:56.351 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.352 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:14:56.353 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:14:56.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:14:56.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:14:56.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:14:56.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-24T17:14:56.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:56.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:14:56.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:14:56.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:14:56.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:14:56.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-24T17:14:56.374 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:14:56.374 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:56.374 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:56.374 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:56.374 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.374 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.375 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:56.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-24T17:14:56.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:14:56.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:14:56.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:56.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:56.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:14:56.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:56.404 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:56.405 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:14:56.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:14:56.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:14:56.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:14:56.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:14:56.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:14:56.460 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:14:56.460 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:56.460 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:56.461 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:56.461 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.461 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.461 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:56.462 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:14:56.462 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:14:56.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:14:56.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:56.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:14:56.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:14:56.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:14:56.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:14:56.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:14:56.629 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:14:56.629 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:56.629 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:56.629 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:56.629 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.629 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.629 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:56.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:56.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:14:56.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-24T17:14:56.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:14:56.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:14:56.650 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:14:56.650 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.650 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:14:56.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:14:56.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:14:56.703 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:14:56.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:14:56.714 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:14:56.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:14:56.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-24T17:14:56.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:14:56.716 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:14:56.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=20cd5e43-a08d-4824-acb0-2ca8a86fc591 2026-02-24T17:14:56.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 20cd5e43-a08d-4824-acb0-2ca8a86fc591' 2026-02-24T17:14:56.717 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 20cd5e43-a08d-4824-acb0-2ca8a86fc591 2026-02-24T17:14:56.717 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:14:56.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCQ3J1pneDvKhAA+HOVNsD9HqKxP4dQPuA43Q== 2026-02-24T17:14:56.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCQ3J1pneDvKhAA+HOVNsD9HqKxP4dQPuA43Q=="}' 2026-02-24T17:14:56.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 20cd5e43-a08d-4824-acb0-2ca8a86fc591 -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:56.836 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:56.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:14:56.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQCQ3J1pneDvKhAA+HOVNsD9HqKxP4dQPuA43Q== --osd-uuid 20cd5e43-a08d-4824-acb0-2ca8a86fc591 2026-02-24T17:14:56.861 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:56.853+0000 7f05ce937a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:56.863 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:56.853+0000 7f05ce937a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:56.864 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:56.853+0000 7f05ce937a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:56.864 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:56.857+0000 7f05ce937a40 -1 bdev(0x55f04323e400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:14:56.864 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:56.857+0000 7f05ce937a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:14:57.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:14:57.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:14:57.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:14:57.589 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:14:57.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:14:57.712 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:14:57.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:14:57.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-24T17:14:57.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:14:57.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:14:57.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:14:57.737 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:57.729+0000 7f107053aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:57.739 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:57.729+0000 7f107053aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:57.741 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:57.733+0000 7f107053aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:14:57.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:14:57.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:57.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:57.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:14:58.158 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:58.149+0000 7f107053aa40 -1 Falling back to public interface 2026-02-24T17:14:58.869 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:14:58.861+0000 7f107053aa40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:14:58.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:14:58.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:14:58.935 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:14:58.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:14:58.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:14:58.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:14:59.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:00.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:00.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:00.154 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:00.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:00.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:00.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:00.369 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/327258249,v1:127.0.0.1:6801/327258249] [v2:127.0.0.1:6802/327258249,v1:127.0.0.1:6803/327258249] exists,up 20cd5e43-a08d-4824-acb0-2ca8a86fc591 2026-02-24T17:15:00.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:00.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:00.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:00.372 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-24T17:15:00.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:00.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:00.373 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:00.373 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.373 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:00.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:15:00.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:00.425 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:00.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:00.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:00.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:00.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:00.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:00.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:00.434 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:00.434 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:00.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:00.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:00.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:00.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:00.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:00.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:00.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:00.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:00.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:00.547 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:00.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:00.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:00.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:00.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:00.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:00.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:00.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:00.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:00.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:00.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:00.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:00.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-24T17:15:00.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-24T17:15:00.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:00.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:00.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:00.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:00.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:00.560 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:00.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:00.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:00.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:00.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:00.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:00.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:00.563 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:00.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:00.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:00.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:00.566 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:00.566 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.566 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:00.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:00.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:00.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:00.568 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:00.568 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.568 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.568 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:00.570 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:00.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:00.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:00.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-24T17:15:00.591 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:00.591 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:00.591 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:00.592 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:00.592 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.592 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:00.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-24T17:15:00.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:00.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:00.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:00.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:00.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:00.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:00.624 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:00.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:00.683 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:00.683 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:00.683 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:00.683 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:00.683 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:00.684 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:00.684 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:00.684 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:00.684 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:00.684 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.684 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.684 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:00.685 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:00.685 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:00.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:00.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:00.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:00.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:00.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:00.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:00.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:00.850 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:00.850 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:00.850 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:00.850 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:00.850 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.850 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.850 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:00.851 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:00.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:00.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-24T17:15:00.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:00.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:00.870 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:00.870 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.870 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:00.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:15:00.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:00.919 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-24T17:15:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:00.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:00.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=27e55f81-5e9e-45d4-af8b-ed94bf2d36b8 2026-02-24T17:15:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 27e55f81-5e9e-45d4-af8b-ed94bf2d36b8' 2026-02-24T17:15:00.934 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 27e55f81-5e9e-45d4-af8b-ed94bf2d36b8 2026-02-24T17:15:00.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:00.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCU3J1ptmnKNxAAVApKbjy0RpC2qDNG3R5f6g== 2026-02-24T17:15:00.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCU3J1ptmnKNxAAVApKbjy0RpC2qDNG3R5f6g=="}' 2026-02-24T17:15:00.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 27e55f81-5e9e-45d4-af8b-ed94bf2d36b8 -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:01.049 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:01.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:01.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQCU3J1ptmnKNxAAVApKbjy0RpC2qDNG3R5f6g== --osd-uuid 27e55f81-5e9e-45d4-af8b-ed94bf2d36b8 2026-02-24T17:15:01.073 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.065+0000 7efc3dde7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:01.074 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.065+0000 7efc3dde7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:01.075 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.065+0000 7efc3dde7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:01.076 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.065+0000 7efc3dde7a40 -1 bdev(0x5646bbd64400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:01.076 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.069+0000 7efc3dde7a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:01.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:01.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:01.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:01.857 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:01.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:01.978 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:01.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:01.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-24T17:15:01.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:01.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:01.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:02.000 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.993+0000 7f69e7b7ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:02.003 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.993+0000 7f69e7b7ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:02.004 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:01.997+0000 7f69e7b7ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:02.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:02.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:02.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:02.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:02.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:02.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:02.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:02.089 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:02.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:02.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:02.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:02.562 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:02.553+0000 7f69e7b7ea40 -1 Falling back to public interface 2026-02-24T17:15:03.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:03.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:03.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:03.199 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:03.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:03.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:03.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:03.456 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:03.449+0000 7f69e7b7ea40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:04.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:04.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:04.410 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:04.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:04.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:04.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:04.621 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3532400090,v1:127.0.0.1:6801/3532400090] [v2:127.0.0.1:6802/3532400090,v1:127.0.0.1:6803/3532400090] exists,up 27e55f81-5e9e-45d4-af8b-ed94bf2d36b8 2026-02-24T17:15:04.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:04.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-24T17:15:04.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:04.677 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:04.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:04.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:04.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:04.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:04.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:04.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:04.688 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:04.688 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:04.689 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:04.689 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:04.689 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:04.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:04.996 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:04.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:04.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:04.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:04.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:04.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:05.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:05.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:05.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:05.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:05.003 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:05.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:05.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:05.008 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:05.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:05.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:05.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:05.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:05.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:05.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:05.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:05.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:05.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:05.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:05.014 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:05.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:05.014 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:05.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:05.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:05.016 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:05.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:05.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:05.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:05.018 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:05.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:05.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:05.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:05.020 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:05.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:05.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:05.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:05.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:05.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:05.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:05.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:05.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:05.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:05.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:05.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-24T17:15:05.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:05.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:05.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:05.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:05.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:05.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-24T17:15:05.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:05.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:05.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:05.048 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:05.048 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.048 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:05.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:15:05.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:05.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:05.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:05.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:05.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:05.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:05.077 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:05.078 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:05.134 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.135 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.136 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:05.136 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:05.136 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:05.191 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-24T17:15:05.191 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:05.191 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:05.191 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:05.191 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.192 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.192 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:05.192 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-24T17:15:05.192 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:05.248 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:05.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:05.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:05.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:05.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:05.378 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:05.378 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.378 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:05.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:05.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:05.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-24T17:15:05.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:05.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:05.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:05.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:05.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:05.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:05.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:05.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:05.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:05.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:05.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:05.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:05.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:05.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:05.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:05.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:05.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:05.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:05.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:05.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:05.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:05.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:05.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:05.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:05.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:05.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:05.404 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:05.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bcb9740f-06b7-4aca-8e08-b93e25d93fe2 2026-02-24T17:15:05.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bcb9740f-06b7-4aca-8e08-b93e25d93fe2' 2026-02-24T17:15:05.404 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 bcb9740f-06b7-4aca-8e08-b93e25d93fe2 2026-02-24T17:15:05.405 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:05.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZ3J1p3vw0GBAAtF65T9iFmAj/3TQOLxjY4w== 2026-02-24T17:15:05.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZ3J1p3vw0GBAAtF65T9iFmAj/3TQOLxjY4w=="}' 2026-02-24T17:15:05.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bcb9740f-06b7-4aca-8e08-b93e25d93fe2 -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:05.521 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:05.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:05.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZ3J1p3vw0GBAAtF65T9iFmAj/3TQOLxjY4w== --osd-uuid bcb9740f-06b7-4aca-8e08-b93e25d93fe2 2026-02-24T17:15:05.544 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:05.533+0000 7f756eee8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:05.545 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:05.537+0000 7f756eee8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:05.546 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:05.537+0000 7f756eee8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:05.546 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:05.537+0000 7f756eee8a40 -1 bdev(0x55b977aa8400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:05.547 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:05.537+0000 7f756eee8a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:06.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:06.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:06.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:06.299 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:06.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:06.425 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:06.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:06.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:06.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:06.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:06.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:06.448 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:06.437+0000 7f55a4f43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:06.450 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:06.441+0000 7f55a4f43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:06.452 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:06.441+0000 7f55a4f43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:06.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:06.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:06.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:06.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:06.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:06.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:06.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:06.538 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:06.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:06.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:06.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:06.994 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:06.985+0000 7f55a4f43a40 -1 Falling back to public interface 2026-02-24T17:15:07.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:07.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:07.648 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:07.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:07.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:07.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:07.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:07.943 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:07.933+0000 7f55a4f43a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:08.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:08.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:08.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:08.760 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:08.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:08.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:08.976 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2248520404,v1:127.0.0.1:6801/2248520404] [v2:127.0.0.1:6802/2248520404,v1:127.0.0.1:6803/2248520404] exists,up bcb9740f-06b7-4aca-8e08-b93e25d93fe2 2026-02-24T17:15:08.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:08.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:08.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:08.978 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-24T17:15:08.978 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:08.978 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:08.978 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:08.978 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:08.978 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:08.978 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:08.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-24T17:15:08.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:09.034 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:09.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:09.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:09.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:09.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:09.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:09.152 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:09.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:09.153 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:09.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:09.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:09.155 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:09.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:09.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:09.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:09.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:09.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:09.159 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:09.159 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:09.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:09.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:09.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:09.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:09.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:09.167 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:09.168 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:09.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:09.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:09.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:09.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:09.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:09.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:09.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:09.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:09.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:09.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:09.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:09.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:09.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:09.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:09.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:09.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:09.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:09.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:09.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:09.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:09.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:09.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:09.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:09.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:09.180 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:09.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:09.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:09.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:09.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-24T17:15:09.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:09.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:09.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:09.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:09.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:09.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-24T17:15:09.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:09.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:09.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:09.204 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:09.204 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.204 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:09.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-24T17:15:09.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:09.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:09.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:09.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:09.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:09.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:09.235 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:09.235 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:09.235 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:09.235 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:09.235 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.235 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.236 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:09.236 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:09.236 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:09.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:09.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:09.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:09.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:09.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:09.290 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:09.290 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:09.290 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:09.290 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:09.290 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.291 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.291 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:09.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:09.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:09.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:09.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:09.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:09.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:09.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:09.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:09.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:09.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:09.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:09.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:09.458 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:09.459 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.459 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:09.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:09.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:09.478 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-24T17:15:09.478 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:09.478 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:09.478 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:09.479 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.479 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.479 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:09.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:09.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:09.528 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:09.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-24T17:15:09.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:09.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:09.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:09.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:09.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:09.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:09.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:09.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:09.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:09.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:09.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:09.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:09.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-24T17:15:09.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:09.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:09.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=179cf26c-8c82-476a-801c-06c88d6ee7b5 2026-02-24T17:15:09.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 179cf26c-8c82-476a-801c-06c88d6ee7b5' 2026-02-24T17:15:09.543 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 179cf26c-8c82-476a-801c-06c88d6ee7b5 2026-02-24T17:15:09.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:09.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCd3J1pKkCOIBAAAMGmxHcfwOJhKOE5Xpn+4A== 2026-02-24T17:15:09.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCd3J1pKkCOIBAAAMGmxHcfwOJhKOE5Xpn+4A=="}' 2026-02-24T17:15:09.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 179cf26c-8c82-476a-801c-06c88d6ee7b5 -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:09.660 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:09.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:09.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQCd3J1pKkCOIBAAAMGmxHcfwOJhKOE5Xpn+4A== --osd-uuid 179cf26c-8c82-476a-801c-06c88d6ee7b5 2026-02-24T17:15:09.685 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:09.677+0000 7f7be2bd4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:09.686 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:09.677+0000 7f7be2bd4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:09.687 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:09.677+0000 7f7be2bd4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:09.687 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:09.677+0000 7f7be2bd4a40 -1 bdev(0x556444254400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:09.688 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:09.677+0000 7f7be2bd4a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:10.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:10.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:10.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:10.409 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:10.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:10.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:10.521 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:10.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-24T17:15:10.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:10.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:10.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:10.534 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:10.525+0000 7f9b75bd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:10.535 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:10.525+0000 7f9b75bd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:10.536 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:10.529+0000 7f9b75bd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:10.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:10.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:10.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:10.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:10.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:10.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:10.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:10.636 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:10.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:10.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:10.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:11.018 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:11.009+0000 7f9b75bd1a40 -1 Falling back to public interface 2026-02-24T17:15:11.692 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:11.681+0000 7f9b75bd1a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:11.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:11.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:11.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:11.746 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:11.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:11.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:11.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:12.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:12.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:12.859 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:12.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:12.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:12.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:13.076 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1882917445,v1:127.0.0.1:6801/1882917445] [v2:127.0.0.1:6802/1882917445,v1:127.0.0.1:6803/1882917445] exists,up 179cf26c-8c82-476a-801c-06c88d6ee7b5 2026-02-24T17:15:13.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:13.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:13.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:13.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:13.141 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:13.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:13.149 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:09.213+0000 7f14c7a10ec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-24T17:15:13.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:13.150 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:11.009+0000 7f9b75bd1a40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-24T17:15:13.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:13.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:13.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:13.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:13.152 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:13.152 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:13.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:13.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:13.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:13.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:13.259 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:13.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:13.260 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:13.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:13.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:13.262 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:13.263 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:13.263 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:13.264 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:13.264 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:13.265 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:13.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:13.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:13.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:13.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:13.272 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:13.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:13.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:13.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:13.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:13.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:13.274 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:13.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:13.275 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:13.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:13.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:13.276 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:13.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:13.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:13.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:13.278 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:13.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:13.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:13.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:13.282 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:13.282 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.282 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:13.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:13.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:13.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:13.284 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:13.284 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.284 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.284 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:13.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:13.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:13.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:13.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-24T17:15:13.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:13.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:13.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:13.308 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:13.308 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.309 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:13.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-24T17:15:13.334 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:13.335 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:13.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:13.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:13.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:13.337 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:13.337 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:13.337 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:13.337 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:13.337 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:13.338 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.338 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.338 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:13.338 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:13.338 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:13.393 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:13.393 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:13.393 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:13.393 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:13.393 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:13.393 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:13.394 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:13.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:13.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:13.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:13.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:13.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:13.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:13.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:13.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:13.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:13.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:13.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:13.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:13.563 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:13.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:13.581 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:13.636 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:13.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-24T17:15:13.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:13.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:13.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:13.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:13.646 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:13.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:13.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:13.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:13.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-24T17:15:13.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:13.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:13.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=52a71234-a305-4b37-bdc0-34e314c37a5c 2026-02-24T17:15:13.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 52a71234-a305-4b37-bdc0-34e314c37a5c' 2026-02-24T17:15:13.649 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 52a71234-a305-4b37-bdc0-34e314c37a5c 2026-02-24T17:15:13.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:13.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCh3J1p9cPYJhAARDR5BY5enc08Vs8vlzCnlg== 2026-02-24T17:15:13.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCh3J1p9cPYJhAARDR5BY5enc08Vs8vlzCnlg=="}' 2026-02-24T17:15:13.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 52a71234-a305-4b37-bdc0-34e314c37a5c -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:13.765 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:13.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:13.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQCh3J1p9cPYJhAARDR5BY5enc08Vs8vlzCnlg== --osd-uuid 52a71234-a305-4b37-bdc0-34e314c37a5c 2026-02-24T17:15:13.789 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:13.781+0000 7fb0383cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:13.790 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:13.781+0000 7fb0383cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:13.791 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:13.781+0000 7fb0383cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:13.792 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:13.781+0000 7fb0383cca40 -1 bdev(0x564a30764400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:13.792 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:13.781+0000 7fb0383cca40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:14.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:14.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:14.513 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:14.635 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:14.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:14.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-24T17:15:14.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:14.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:14.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:14.657 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:14.649+0000 7f6e6e95aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:14.659 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:14.649+0000 7f6e6e95aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:14.660 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:14.653+0000 7f6e6e95aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:14.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:14.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:14.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:14.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:14.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:14.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:14.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:14.747 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:14.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:14.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:14.855 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:15.066 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:15.057+0000 7f6e6e95aa40 -1 Falling back to public interface 2026-02-24T17:15:15.763 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:15.753+0000 7f6e6e95aa40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:15.855 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:15.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:15.856 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:15.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:15.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:15.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:15.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:16.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:16.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:16.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:16.968 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:16.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:16.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:17.180 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3919087036,v1:127.0.0.1:6801/3919087036] [v2:127.0.0.1:6802/3919087036,v1:127.0.0.1:6803/3919087036] exists,up 52a71234-a305-4b37-bdc0-34e314c37a5c 2026-02-24T17:15:17.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:17.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:17.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:17.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-24T17:15:17.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:17.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:17.182 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:17.182 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.182 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.182 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:17.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:17.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:17.238 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:17.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:17.245 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:13.317+0000 7fce50c81ec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-24T17:15:17.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:17.246 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:15.057+0000 7f6e6e95aa40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:17.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:17.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:17.356 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:17.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:17.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:17.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:17.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:17.359 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:17.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:17.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:17.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:17.361 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:17.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:17.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:17.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:17.367 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:17.367 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.367 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:17.371 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:17.371 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:17.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:17.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:17.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:17.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:17.372 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:17.373 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:17.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:17.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:17.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:17.375 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:17.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:17.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:17.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:17.377 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:17.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:17.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:17.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:17.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:17.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:17.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:17.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:17.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:17.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:17.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:17.385 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:17.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:17.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:17.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:17.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-24T17:15:17.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:17.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:17.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:17.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:17.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:17.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-24T17:15:17.408 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:17.408 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:17.408 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:17.408 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:17.408 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.408 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.408 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:17.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-24T17:15:17.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:17.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:17.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:17.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:17.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:17.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:17.438 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:17.438 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:17.439 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:17.439 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:17.439 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.439 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.439 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:17.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:17.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:17.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:17.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:17.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:17.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:17.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:17.493 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:17.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:17.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:17.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:17.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:17.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:17.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:17.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:17.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:17.660 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:17.660 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:17.660 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:17.660 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:17.660 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.660 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.660 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:17.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:17.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:17.680 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-24T17:15:17.681 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:17.681 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:17.681 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:17.681 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.681 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.681 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:17.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:17.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:17.734 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:17.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:17.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:17.743 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:17.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bdbdc77b-4d67-45c5-a4da-a2233d055c0e 2026-02-24T17:15:17.744 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 bdbdc77b-4d67-45c5-a4da-a2233d055c0e 2026-02-24T17:15:17.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bdbdc77b-4d67-45c5-a4da-a2233d055c0e' 2026-02-24T17:15:17.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:17.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCl3J1pB1PBLBAALwNo2Xad6AK0A7r5MTL7Zw== 2026-02-24T17:15:17.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCl3J1pB1PBLBAALwNo2Xad6AK0A7r5MTL7Zw=="}' 2026-02-24T17:15:17.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bdbdc77b-4d67-45c5-a4da-a2233d055c0e -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:17.870 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:17.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:17.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQCl3J1pB1PBLBAALwNo2Xad6AK0A7r5MTL7Zw== --osd-uuid bdbdc77b-4d67-45c5-a4da-a2233d055c0e 2026-02-24T17:15:17.894 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:17.885+0000 7f27a9ac4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:17.896 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:17.885+0000 7f27a9ac4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:17.897 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:17.889+0000 7f27a9ac4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:17.898 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:17.889+0000 7f27a9ac4a40 -1 bdev(0x564a5db32400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:17.898 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:17.889+0000 7f27a9ac4a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:18.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:18.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:18.650 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:18.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:18.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:18.771 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:18.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:18.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-24T17:15:18.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:18.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:18.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:18.795 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:18.785+0000 7fcbb087ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:18.798 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:18.789+0000 7fcbb087ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:18.799 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:18.789+0000 7fcbb087ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:18.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:18.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:18.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:18.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:18.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:18.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:18.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:18.880 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:18.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:18.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:18.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:19.234 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:19.229+0000 7fcbb087ea40 -1 Falling back to public interface 2026-02-24T17:15:19.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:19.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:19.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:19.987 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:19.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:19.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:20.162 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:20.157+0000 7fcbb087ea40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:20.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:21.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:21.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:21.207 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:21.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:21.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:21.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:21.411 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1356003286,v1:127.0.0.1:6801/1356003286] [v2:127.0.0.1:6802/1356003286,v1:127.0.0.1:6803/1356003286] exists,up bdbdc77b-4d67-45c5-a4da-a2233d055c0e 2026-02-24T17:15:21.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:21.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:21.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:21.411 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-24T17:15:21.411 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:21.412 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:21.412 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:21.412 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.412 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.412 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:21.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:21.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:21.468 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:21.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:21.475 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:17.417+0000 7fd76ab1dec0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-24T17:15:21.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:21.477 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:19.229+0000 7fcbb087ea40 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-24T17:15:21.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:21.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:21.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:21.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:21.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:21.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:21.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:21.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:21.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:21.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:21.587 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:21.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:21.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:21.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:21.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:21.589 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:21.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:21.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:21.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:21.591 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:21.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:21.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:21.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:21.596 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:21.596 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.596 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:21.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:21.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:21.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:21.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:21.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:21.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:21.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:21.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:21.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:21.603 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:21.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:21.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:21.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:21.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:21.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:21.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:21.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:21.608 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:21.609 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.609 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:21.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:21.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:21.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:21.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:21.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:21.613 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:21.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-24T17:15:21.635 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:21.636 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:21.636 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:21.636 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:21.636 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.636 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.636 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:21.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-24T17:15:21.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:21.666 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:21.667 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:21.667 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.667 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.667 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:21.667 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:21.667 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:21.722 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:21.723 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:21.723 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.723 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.723 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:21.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:21.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:21.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:21.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:21.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:21.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:21.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:21.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:21.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:21.890 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:21.890 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:21.890 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:21.890 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:21.890 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.891 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.891 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:21.891 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:21.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:21.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-24T17:15:21.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:21.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:21.910 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:21.910 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.910 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:21.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:21.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:21.963 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:21.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-24T17:15:21.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:21.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:21.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:21.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:21.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:21.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:21.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:21.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:21.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:21.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:21.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:21.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:21.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:21.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:21.980 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:21.980 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:21.980 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:21.981 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:21.981 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:21.981 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:21.981 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:21.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:21.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:21.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:21.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:21.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:21.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:21.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:21.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:21.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:21.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:21.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:21.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-24T17:15:21.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:21.983 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:21.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=90fb87ba-16bf-49a3-be93-f77fa250ab6b 2026-02-24T17:15:21.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 90fb87ba-16bf-49a3-be93-f77fa250ab6b' 2026-02-24T17:15:21.984 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 90fb87ba-16bf-49a3-be93-f77fa250ab6b 2026-02-24T17:15:21.986 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:21.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCp3J1pnj18OhAA8eMQrsyffb3GwsE+L6kXnQ== 2026-02-24T17:15:21.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCp3J1pnj18OhAA8eMQrsyffb3GwsE+L6kXnQ=="}' 2026-02-24T17:15:21.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 90fb87ba-16bf-49a3-be93-f77fa250ab6b -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:22.099 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:22.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:22.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQCp3J1pnj18OhAA8eMQrsyffb3GwsE+L6kXnQ== --osd-uuid 90fb87ba-16bf-49a3-be93-f77fa250ab6b 2026-02-24T17:15:22.123 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:22.117+0000 7fa8335fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:22.125 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:22.121+0000 7fa8335fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:22.125 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:22.121+0000 7fa8335fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:22.126 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:22.121+0000 7fa8335fea40 -1 bdev(0x55878e5aa400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:22.126 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:22.121+0000 7fa8335fea40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:22.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:22.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:22.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:22.886 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:22.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:23.006 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:23.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:23.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-24T17:15:23.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:23.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:23.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:23.030 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:23.025+0000 7f70ea343a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:23.033 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:23.029+0000 7f70ea343a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:23.035 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:23.029+0000 7f70ea343a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:23.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:23.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:23.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:23.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:23.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:23.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:23.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:23.118 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:23.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:23.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:23.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:23.466 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:23.461+0000 7f70ea343a40 -1 Falling back to public interface 2026-02-24T17:15:24.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:24.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:24.232 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:24.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:24.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:24.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:24.403 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:24.397+0000 7f70ea343a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:24.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:25.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:25.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:25.440 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:25.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:25.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:25.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:25.647 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1769450528,v1:127.0.0.1:6801/1769450528] [v2:127.0.0.1:6802/1769450528,v1:127.0.0.1:6803/1769450528] exists,up 90fb87ba-16bf-49a3-be93-f77fa250ab6b 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:25.648 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:25.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:25.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:25.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:25.711 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:25.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:25.717 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:21.649+0000 7f0c9fecaec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-24T17:15:25.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:25.719 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:23.461+0000 7f70ea343a40 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-24T17:15:25.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:25.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:25.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:25.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:25.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:25.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:25.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:25.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:25.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:25.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:25.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:25.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:25.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:25.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:25.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:25.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:25.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:25.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:25.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:25.833 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:25.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:25.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:25.835 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:25.839 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:25.839 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:25.839 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:25.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:25.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:25.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:25.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:25.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:25.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:25.843 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:25.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:25.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:25.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:25.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:25.846 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:25.847 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:25.847 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:25.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:25.848 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:25.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:25.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:25.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:25.851 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:25.851 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:25.851 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:25.851 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:25.854 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:25.854 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:25.854 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:25.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:25.855 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:25.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-24T17:15:25.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:25.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:25.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:25.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:25.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:25.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:25.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:25.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-24T17:15:25.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:25.909 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:25.910 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:25.910 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:25.910 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:25.910 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:25.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:25.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:25.967 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:25.968 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:25.968 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:25.968 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:25.968 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:25.968 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:25.968 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:26.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:26.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:26.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:26.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:26.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:26.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:26.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:26.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:26.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:26.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:26.140 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:26.141 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:26.141 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:26.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:26.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:26.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:26.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:26.208 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:26.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:26.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:26.219 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:26.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:26.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:26.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:26.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-24T17:15:26.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:26.221 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:26.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=91802089-4f47-4367-882f-d70ecfac989b 2026-02-24T17:15:26.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 91802089-4f47-4367-882f-d70ecfac989b' 2026-02-24T17:15:26.222 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 91802089-4f47-4367-882f-d70ecfac989b 2026-02-24T17:15:26.222 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:26.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCu3J1p5b9qDRAAXb8/ppIcIQloK071aKDwcA== 2026-02-24T17:15:26.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCu3J1p5b9qDRAAXb8/ppIcIQloK071aKDwcA=="}' 2026-02-24T17:15:26.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 91802089-4f47-4367-882f-d70ecfac989b -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:26.341 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:26.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:26.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQCu3J1p5b9qDRAAXb8/ppIcIQloK071aKDwcA== --osd-uuid 91802089-4f47-4367-882f-d70ecfac989b 2026-02-24T17:15:26.366 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:26.361+0000 7f4226063a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:26.368 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:26.361+0000 7f4226063a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:26.369 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:26.365+0000 7f4226063a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:26.369 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:26.365+0000 7f4226063a40 -1 bdev(0x56311d9d2400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:26.369 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:26.365+0000 7f4226063a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:27.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:27.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:27.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:27.110 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:27.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:27.237 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:27.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:27.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-24T17:15:27.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:27.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:27.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:27.260 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:27.257+0000 7f794c587a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:27.262 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:27.257+0000 7f794c587a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:27.264 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:27.257+0000 7f794c587a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:27.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:27.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:27.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:27.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:27.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:27.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:27.351 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:27.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:27.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:27.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:27.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:27.786 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:27.781+0000 7f794c587a40 -1 Falling back to public interface 2026-02-24T17:15:28.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:28.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:28.461 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:28.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:28.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:28.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:28.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:28.717 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:28.712+0000 7f794c587a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:29.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:29.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:29.678 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:29.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:29.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:29.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:29.890 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2581824665,v1:127.0.0.1:6801/2581824665] [v2:127.0.0.1:6802/2581824665,v1:127.0.0.1:6803/2581824665] exists,up 91802089-4f47-4367-882f-d70ecfac989b 2026-02-24T17:15:29.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:29.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:29.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:29.957 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:29.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:29.965 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:25.893+0000 7f5e78946ec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-24T17:15:29.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:29.966 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:27.781+0000 7f794c587a40 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-24T17:15:29.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:29.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:30.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:30.077 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:30.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:30.077 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:30.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:30.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:30.078 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:30.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:30.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:30.080 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:30.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:30.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:30.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:30.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:30.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:30.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:30.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:30.092 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:30.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:30.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:30.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:30.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:30.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:30.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:30.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:30.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:30.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:30.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:30.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:30.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:30.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:30.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:30.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:30.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:30.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:30.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:30.099 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:30.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:30.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:30.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:30.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-24T17:15:30.102 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:30.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.11430 2026-02-24T17:15:30.104 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:30.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-24T17:15:30.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-24T17:15:30.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:30.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:30.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:30.127 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:30.127 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.127 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.128 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:30.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-24T17:15:30.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:30.157 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:30.158 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:30.158 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.158 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.158 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:30.158 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:30.158 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get fsid 2026-02-24T17:15:30.212 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.213 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.214 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:30.214 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:30.214 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.11430/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:30.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-24T17:15:30.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:30.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:30.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:30.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:30.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-24T17:15:30.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:30.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:30.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:30.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:30.379 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:30.379 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.379 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:30.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:30.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:30.399 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-24T17:15:30.399 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:30.399 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:30.399 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:30.399 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.400 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-mon.a.asok 2026-02-24T17:15:30.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:30.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-mon.a.asok log flush 2026-02-24T17:15:30.448 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:30.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-24T17:15:30.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-24T17:15:30.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:30.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-24T17:15:30.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:30.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2a09ad01-95c0-4bce-adea-b4e1883e6d8a 2026-02-24T17:15:30.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2a09ad01-95c0-4bce-adea-b4e1883e6d8a' 2026-02-24T17:15:30.461 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 2a09ad01-95c0-4bce-adea-b4e1883e6d8a 2026-02-24T17:15:30.462 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:30.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCy3J1puOPEGxAAeG8gsKp2GClt8m+lko8cag== 2026-02-24T17:15:30.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCy3J1puOPEGxAAeG8gsKp2GClt8m+lko8cag=="}' 2026-02-24T17:15:30.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2a09ad01-95c0-4bce-adea-b4e1883e6d8a -i td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:30.582 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:30.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-24T17:15:30.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQCy3J1puOPEGxAAeG8gsKp2GClt8m+lko8cag== --osd-uuid 2a09ad01-95c0-4bce-adea-b4e1883e6d8a 2026-02-24T17:15:30.606 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:30.600+0000 7fea8c588a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:30.608 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:30.600+0000 7fea8c588a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:30.609 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:30.604+0000 7fea8c588a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:30.610 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:30.604+0000 7fea8c588a40 -1 bdev(0x555a3c60e400 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:30.610 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:30.604+0000 7fea8c588a40 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-24T17:15:31.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-24T17:15:31.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:31.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:31.409 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:31.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:31.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:31.527 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:31.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8a2aea51-6e90-4680-945f-329a32ea08cf --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.11430/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-24T17:15:31.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:31.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:31.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:31.550 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:31.544+0000 7f0fe383da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:31.553 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:31.548+0000 7f0fe383da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:31.554 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:31.548+0000 7f0fe383da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:31.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:31.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:31.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:31.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:31.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:31.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:31.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:31.634 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:31.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:31.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:31.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:32.078 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:32.072+0000 7f0fe383da40 -1 Falling back to public interface 2026-02-24T17:15:32.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:32.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:32.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:32.744 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:32.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:32.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:32.950 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:32.973 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:32.968+0000 7f0fe383da40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:33.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:33.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:33.953 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:33.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:33.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:33.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:34.160 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3522460097,v1:127.0.0.1:6801/3522460097] [v2:127.0.0.1:6802/3522460097,v1:127.0.0.1:6803/3522460097] exists,up 2a09ad01-95c0-4bce-adea-b4e1883e6d8a 2026-02-24T17:15:34.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:34.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:34.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:34.162 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-24T17:15:34.162 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:15:34.162 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:15:34.162 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:34.163 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.163 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:34.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.11430/ceph-osd.0.asok 2026-02-24T17:15:34.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-24T17:15:34.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.11430/ceph-osd.0.asok log flush 2026-02-24T17:15:34.224 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:34.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-24T17:15:34.231 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:30.136+0000 7fc01d4c9ec0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-24T17:15:34.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-24T17:15:34.233 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:32.072+0000 7f0fe383da40 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-24T17:15:34.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:34.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:34.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:34.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:34.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:34.343 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:34.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:34.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:34.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:34.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:34.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:34.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:34.346 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:34.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:34.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:34.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:34.352 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:34.352 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.352 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:34.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/test-erasure-code-plugins 0 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-24T17:15:34.355 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:34.356 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:34.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:34.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:34.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:34.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:34.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:34.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:34.358 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:34.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:34.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:34.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:34.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:34.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:34.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:34.362 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:34.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:34.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-24T17:15:34.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-24T17:15:34.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:34.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.11430 2026-02-24T17:15:34.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.11430 2026-02-24T17:15:34.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:34.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:34.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-24T17:15:34.367 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T17:15:34.368 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T17:15:34.418 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-24T17:15:34.418 DEBUG:teuthology.orchestra.run.trial066:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-24T17:15:34.469 INFO:tasks.workunit.client.0.trial066.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T17:15:34.473 INFO:tasks.workunit.client.0.trial066.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T17:15:34.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/test-erasure-code 2026-02-24T17:15:34.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-24T17:15:34.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-24T17:15:34.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-24T17:15:34.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-24T17:15:34.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-24T17:15:34.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-24T17:15:34.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none ' 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:15:34.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:15:34.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:15:34.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:15:34.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:15:34.478 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:15:34.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:15:34.479 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:15:34.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:15:34.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:15:34.480 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:15:34.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:34.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:15:34.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:15:34.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:15:34.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:15:34.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:15:34.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-24T17:15:34.485 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:15:34.485 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.485 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.45680 2026-02-24T17:15:34.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:15:34.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:15:34.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-24T17:15:34.488 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:15:34.488 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.488 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.489 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.45680 2026-02-24T17:15:34.490 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:15:34.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T17:15:34.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T17:15:34.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:15:34.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-24T17:15:34.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-24T17:15:34.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-24T17:15:34.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:15:34.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:15:34.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:15:34.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-24T17:15:34.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-24T17:15:34.514 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:15:34.514 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:34.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:34.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:34.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:34.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:15:34.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:15:34.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:15:34.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:34.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:34.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:15:34.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.45680/ceph-mon.a.asok 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:34.546 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.45680/ceph-mon.a.asok config get fsid 2026-02-24T17:15:34.600 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:15:34.600 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:15:34.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:15:34.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:15:34.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:15:34.601 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:15:34.601 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:34.601 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:34.602 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:34.602 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.602 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.602 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.45680/ceph-mon.a.asok 2026-02-24T17:15:34.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:15:34.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.45680/ceph-mon.a.asok config get mon_host 2026-02-24T17:15:34.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-24T17:15:34.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-24T17:15:34.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:15:34.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:15:34.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:15:34.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-24T17:15:34.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:15:34.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:15:34.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:34.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:34.774 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:34.774 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.774 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:34.775 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:34.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:15:34.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-24T17:15:34.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:15:34.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:15:34.793 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:15:34.793 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.793 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.45680/ceph-mon.a.asok 2026-02-24T17:15:34.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-24T17:15:34.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.45680/ceph-mon.a.asok log flush 2026-02-24T17:15:34.844 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:15:34.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-24T17:15:34.851 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:34.528+0000 7fd395739ec0 0 load: jerasure load: lrc 2026-02-24T17:15:34.852 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-24T17:15:34.855 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:34.855 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:34.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:34.857 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:34.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:34.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:34.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:34.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-24T17:15:34.859 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:34.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ff4309e5-4781-4561-ba41-623182f211a6 2026-02-24T17:15:34.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ff4309e5-4781-4561-ba41-623182f211a6' 2026-02-24T17:15:34.860 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 ff4309e5-4781-4561-ba41-623182f211a6 2026-02-24T17:15:34.860 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:34.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC23J1pLNeAMxAA7zIcZBgyCzwOdivdDQr2+A== 2026-02-24T17:15:34.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC23J1pLNeAMxAA7zIcZBgyCzwOdivdDQr2+A=="}' 2026-02-24T17:15:34.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ff4309e5-4781-4561-ba41-623182f211a6 -i td/test-erasure-code/0/new.json 2026-02-24T17:15:34.982 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:34.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-24T17:15:34.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC23J1pLNeAMxAA7zIcZBgyCzwOdivdDQr2+A== --osd-uuid ff4309e5-4781-4561-ba41-623182f211a6 2026-02-24T17:15:35.007 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.000+0000 7f0761707a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:35.008 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.000+0000 7f0761707a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:35.009 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.004+0000 7f0761707a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:35.009 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.004+0000 7f0761707a40 -1 bdev(0x558d9067e400 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:35.010 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.004+0000 7f0761707a40 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-24T17:15:35.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-24T17:15:35.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:35.746 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:15:35.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:15:35.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:35.866 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:15:35.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:15:35.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:35.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:35.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:35.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:35.890 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.884+0000 7f7dd4c44a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:35.893 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.888+0000 7f7dd4c44a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:35.895 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:35.888+0000 7f7dd4c44a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:35.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:15:35.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:35.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:15:35.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:35.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:35.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:35.976 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:35.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:35.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:35.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:36.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:36.410 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:36.404+0000 7f7dd4c44a40 -1 Falling back to public interface 2026-02-24T17:15:37.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:37.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:37.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:37.084 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:37.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:37.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:37.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:37.406 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:37.400+0000 7f7dd4c44a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:15:38.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:38.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:38.191 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:38.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:38.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:38.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:15:38.397 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3942389609,v1:127.0.0.1:6801/3942389609] [v2:127.0.0.1:6802/3942389609,v1:127.0.0.1:6803/3942389609] exists,up ff4309e5-4781-4561-ba41-623182f211a6 2026-02-24T17:15:38.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:15:38.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:38.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:38.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:38.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:38.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:38.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:38.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:38.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-24T17:15:38.404 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:38.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c0ad02af-2ba6-4a54-8153-f63a85d7c8ef 2026-02-24T17:15:38.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 c0ad02af-2ba6-4a54-8153-f63a85d7c8ef' 2026-02-24T17:15:38.405 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 c0ad02af-2ba6-4a54-8153-f63a85d7c8ef 2026-02-24T17:15:38.405 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:38.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC63J1p3MRrGBAAJDqEkLbZg6Rp7T0/mTpSHA== 2026-02-24T17:15:38.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC63J1p3MRrGBAAJDqEkLbZg6Rp7T0/mTpSHA=="}' 2026-02-24T17:15:38.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c0ad02af-2ba6-4a54-8153-f63a85d7c8ef -i td/test-erasure-code/1/new.json 2026-02-24T17:15:38.629 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:38.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-24T17:15:38.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC63J1p3MRrGBAAJDqEkLbZg6Rp7T0/mTpSHA== --osd-uuid c0ad02af-2ba6-4a54-8153-f63a85d7c8ef 2026-02-24T17:15:38.655 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:38.648+0000 7f1900875a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:38.656 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:38.652+0000 7f1900875a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:38.657 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:38.652+0000 7f1900875a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:38.658 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:38.652+0000 7f1900875a40 -1 bdev(0x5630a4c2e400 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:38.658 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:38.652+0000 7f1900875a40 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-24T17:15:39.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-24T17:15:39.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:39.360 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:15:39.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:15:39.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:39.642 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:15:39.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:15:39.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:39.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:39.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:39.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:39.666 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:39.660+0000 7f8775ad5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:39.667 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:39.660+0000 7f8775ad5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:39.668 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:39.664+0000 7f8775ad5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:39.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:15:39.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:39.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:15:39.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:39.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:39.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:39.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:39.862 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:39.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:39.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:15:40.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:40.118 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:40.112+0000 7f8775ad5a40 -1 Falling back to public interface 2026-02-24T17:15:40.699 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:40.692+0000 7f8775ad5a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:15:41.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:41.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:41.072 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:41.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:41.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:41.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:15:41.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:42.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:42.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:42.281 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:42.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:42.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:42.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:15:42.493 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2057473276,v1:127.0.0.1:6811/2057473276] [v2:127.0.0.1:6812/2057473276,v1:127.0.0.1:6813/2057473276] exists,up c0ad02af-2ba6-4a54-8153-f63a85d7c8ef 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:42.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:42.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:42.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-24T17:15:42.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:42.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c4a46b09-798a-4cbe-a59b-0c84d1bfe37f 2026-02-24T17:15:42.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c4a46b09-798a-4cbe-a59b-0c84d1bfe37f' 2026-02-24T17:15:42.498 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 c4a46b09-798a-4cbe-a59b-0c84d1bfe37f 2026-02-24T17:15:42.499 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:42.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+3J1pfyE0HhAAHAqiTDSKBMbue74kNOwMhg== 2026-02-24T17:15:42.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+3J1pfyE0HhAAHAqiTDSKBMbue74kNOwMhg=="}' 2026-02-24T17:15:42.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c4a46b09-798a-4cbe-a59b-0c84d1bfe37f -i td/test-erasure-code/2/new.json 2026-02-24T17:15:42.728 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:42.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-24T17:15:42.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+3J1pfyE0HhAAHAqiTDSKBMbue74kNOwMhg== --osd-uuid c4a46b09-798a-4cbe-a59b-0c84d1bfe37f 2026-02-24T17:15:42.754 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:42.748+0000 7f9255c8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:42.755 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:42.748+0000 7f9255c8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:42.756 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:42.752+0000 7f9255c8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:42.757 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:42.752+0000 7f9255c8fa40 -1 bdev(0x55d37e44a400 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:42.757 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:42.752+0000 7f9255c8fa40 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-24T17:15:43.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-24T17:15:43.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:43.420 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:15:43.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:15:43.421 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:43.699 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:15:43.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:15:43.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:43.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:43.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:43.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:43.714 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:43.708+0000 7fa1795c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:43.715 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:43.708+0000 7fa1795c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:43.717 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:43.712+0000 7fa1795c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:43.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:15:43.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:43.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:15:43.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:43.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:43.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:43.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:43.916 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:43.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:43.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:15:44.118 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:44.112+0000 7fa1795c5a40 -1 Falling back to public interface 2026-02-24T17:15:44.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:44.673 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:44.668+0000 7fa1795c5a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:15:45.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:45.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:45.128 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:45.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:45.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:45.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:15:45.342 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1662425514,v1:127.0.0.1:6819/1662425514] [v2:127.0.0.1:6820/1662425514,v1:127.0.0.1:6821/1662425514] exists,up c4a46b09-798a-4cbe-a59b-0c84d1bfe37f 2026-02-24T17:15:45.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:45.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:45.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:45.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:45.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-24T17:15:45.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:45.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:45.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:15:45.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:45.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-24T17:15:45.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:45.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:45.348 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:45.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:45.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:45.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-24T17:15:45.351 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:45.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bdee9d30-7c32-48b2-a655-2f158e55fcf2 2026-02-24T17:15:45.352 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 bdee9d30-7c32-48b2-a655-2f158e55fcf2 2026-02-24T17:15:45.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 bdee9d30-7c32-48b2-a655-2f158e55fcf2' 2026-02-24T17:15:45.353 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:45.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDB3J1pOWcsFRAAPPCRcLvx1te0rF0dv8KnbQ== 2026-02-24T17:15:45.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDB3J1pOWcsFRAAPPCRcLvx1te0rF0dv8KnbQ=="}' 2026-02-24T17:15:45.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bdee9d30-7c32-48b2-a655-2f158e55fcf2 -i td/test-erasure-code/3/new.json 2026-02-24T17:15:45.575 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:15:45.584 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-24T17:15:45.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDB3J1pOWcsFRAAPPCRcLvx1te0rF0dv8KnbQ== --osd-uuid bdee9d30-7c32-48b2-a655-2f158e55fcf2 2026-02-24T17:15:45.601 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:45.596+0000 7f95dc807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:45.602 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:45.596+0000 7f95dc807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:45.603 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:45.596+0000 7f95dc807a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:45.605 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:45.596+0000 7f95dc807a40 -1 bdev(0x55e532338400 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:45.605 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:45.596+0000 7f95dc807a40 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-24T17:15:46.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-24T17:15:46.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:46.300 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:15:46.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:15:46.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:46.592 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:15:46.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:15:46.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:46.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:46.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:46.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:46.616 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:46.608+0000 7f2532cbca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:46.618 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:46.612+0000 7f2532cbca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:46.619 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:46.612+0000 7f2532cbca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:46.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:15:46.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:46.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:15:46.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:46.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:46.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:46.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:46.808 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:46.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:46.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:15:47.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:47.062 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:47.056+0000 7f2532cbca40 -1 Falling back to public interface 2026-02-24T17:15:47.630 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:47.624+0000 7f2532cbca40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:15:48.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:48.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:48.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:48.016 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:48.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:48.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:15:48.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:49.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:49.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:49.227 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:49.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:49.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:49.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:15:49.435 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2698107181,v1:127.0.0.1:6827/2698107181] [v2:127.0.0.1:6828/2698107181,v1:127.0.0.1:6829/2698107181] exists,up bdee9d30-7c32-48b2-a655-2f158e55fcf2 2026-02-24T17:15:49.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:49.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:49.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:49.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:49.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-24T17:15:49.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:49.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:49.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:49.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-24T17:15:49.442 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:49.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b3da5122-d45d-4679-9540-861f361d4131 2026-02-24T17:15:49.442 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 b3da5122-d45d-4679-9540-861f361d4131 2026-02-24T17:15:49.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 b3da5122-d45d-4679-9540-861f361d4131' 2026-02-24T17:15:49.443 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDF3J1pC4SwGhAAmn4AauK6lHJ4jq6T8KDUyw== 2026-02-24T17:15:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDF3J1pC4SwGhAAmn4AauK6lHJ4jq6T8KDUyw=="}' 2026-02-24T17:15:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b3da5122-d45d-4679-9540-861f361d4131 -i td/test-erasure-code/4/new.json 2026-02-24T17:15:49.670 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:15:49.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-24T17:15:49.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDF3J1pC4SwGhAAmn4AauK6lHJ4jq6T8KDUyw== --osd-uuid b3da5122-d45d-4679-9540-861f361d4131 2026-02-24T17:15:49.697 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:49.692+0000 7f67db426a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:49.699 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:49.692+0000 7f67db426a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:49.700 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:49.692+0000 7f67db426a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:49.700 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:49.692+0000 7f67db426a40 -1 bdev(0x55651b3b2400 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:49.700 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:49.696+0000 7f67db426a40 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-24T17:15:50.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-24T17:15:50.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:50.348 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:15:50.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:15:50.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:50.636 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:15:50.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:15:50.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:50.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:50.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:50.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:50.652 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:50.644+0000 7f89ac2f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:50.653 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:50.648+0000 7f89ac2f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:50.654 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:50.648+0000 7f89ac2f2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:50.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:15:50.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:50.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:15:50.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:50.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:50.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:50.860 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:50.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:50.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:50.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:15:51.022 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:51.016+0000 7f89ac2f2a40 -1 Falling back to public interface 2026-02-24T17:15:51.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:51.679 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:51.672+0000 7f89ac2f2a40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:15:52.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:52.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:52.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:52.072 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:52.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:52.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:15:52.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:53.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:53.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:53.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:15:53.289 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:15:53.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:53.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:15:53.495 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1240197238,v1:127.0.0.1:6835/1240197238] [v2:127.0.0.1:6836/1240197238,v1:127.0.0.1:6837/1240197238] exists,up b3da5122-d45d-4679-9540-861f361d4131 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:53.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:53.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-24T17:15:53.499 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a9ded35c-a91d-458a-aca3-7d5b2947ee4b 2026-02-24T17:15:53.500 INFO:tasks.workunit.client.0.trial066.stdout:add osd5 a9ded35c-a91d-458a-aca3-7d5b2947ee4b 2026-02-24T17:15:53.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 a9ded35c-a91d-458a-aca3-7d5b2947ee4b' 2026-02-24T17:15:53.500 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:53.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDJ3J1pf2U+HhAA4n5jVYbokHc7ZNtRh0YPRQ== 2026-02-24T17:15:53.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDJ3J1pf2U+HhAA4n5jVYbokHc7ZNtRh0YPRQ=="}' 2026-02-24T17:15:53.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a9ded35c-a91d-458a-aca3-7d5b2947ee4b -i td/test-erasure-code/5/new.json 2026-02-24T17:15:53.731 INFO:tasks.workunit.client.0.trial066.stdout:5 2026-02-24T17:15:53.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-24T17:15:53.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJ3J1pf2U+HhAA4n5jVYbokHc7ZNtRh0YPRQ== --osd-uuid a9ded35c-a91d-458a-aca3-7d5b2947ee4b 2026-02-24T17:15:53.757 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:53.752+0000 7ff67eca5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:53.759 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:53.752+0000 7ff67eca5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:53.760 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:53.752+0000 7ff67eca5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:53.760 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:53.756+0000 7ff67eca5a40 -1 bdev(0x563a8dca0400 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:53.760 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:53.756+0000 7ff67eca5a40 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-24T17:15:54.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-24T17:15:54.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:54.444 INFO:tasks.workunit.client.0.trial066.stdout:adding osd5 key to auth repository 2026-02-24T17:15:54.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-24T17:15:54.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:54.730 INFO:tasks.workunit.client.0.trial066.stdout:start osd.5 2026-02-24T17:15:54.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-24T17:15:54.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:54.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:54.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:54.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:54.754 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:54.748+0000 7fae7342aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:54.756 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:54.748+0000 7fae7342aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:54.757 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:54.752+0000 7fae7342aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:54.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:54.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:15:55.158 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:55.152+0000 7fae7342aa40 -1 Falling back to public interface 2026-02-24T17:15:55.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:55.746 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:55.740+0000 7fae7342aa40 -1 osd.5 0 log_to_monitors true 2026-02-24T17:15:56.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:56.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:56.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:56.169 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:56.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:56.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:15:56.381 INFO:tasks.workunit.client.0.trial066.stdout:osd.5 up in weight 1 up_from 28 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3969807046,v1:127.0.0.1:6843/3969807046] [v2:127.0.0.1:6844/3969807046,v1:127.0.0.1:6845/3969807046] exists,up a9ded35c-a91d-458a-aca3-7d5b2947ee4b 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:56.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:56.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:56.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:56.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:56.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:56.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:56.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:56.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-24T17:15:56.385 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:56.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1208e7cb-6405-4dcb-b570-39a424e83021 2026-02-24T17:15:56.386 INFO:tasks.workunit.client.0.trial066.stdout:add osd6 1208e7cb-6405-4dcb-b570-39a424e83021 2026-02-24T17:15:56.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 1208e7cb-6405-4dcb-b570-39a424e83021' 2026-02-24T17:15:56.386 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:56.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDM3J1pzHWHFxAAF2FEPiJ5Ak68WHhFdyiMIA== 2026-02-24T17:15:56.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDM3J1pzHWHFxAAF2FEPiJ5Ak68WHhFdyiMIA=="}' 2026-02-24T17:15:56.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1208e7cb-6405-4dcb-b570-39a424e83021 -i td/test-erasure-code/6/new.json 2026-02-24T17:15:56.615 INFO:tasks.workunit.client.0.trial066.stdout:6 2026-02-24T17:15:56.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-24T17:15:56.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDM3J1pzHWHFxAAF2FEPiJ5Ak68WHhFdyiMIA== --osd-uuid 1208e7cb-6405-4dcb-b570-39a424e83021 2026-02-24T17:15:56.641 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:56.636+0000 7f168921ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:56.642 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:56.636+0000 7f168921ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:56.643 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:56.636+0000 7f168921ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:56.644 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:56.636+0000 7f168921ba40 -1 bdev(0x55951b9b0400 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:56.644 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:56.636+0000 7f168921ba40 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-24T17:15:57.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-24T17:15:57.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:15:57.324 INFO:tasks.workunit.client.0.trial066.stdout:adding osd6 key to auth repository 2026-02-24T17:15:57.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-24T17:15:57.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:15:57.614 INFO:tasks.workunit.client.0.trial066.stdout:start osd.6 2026-02-24T17:15:57.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-24T17:15:57.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:15:57.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:15:57.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:15:57.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:15:57.636 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:57.628+0000 7fe37553aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:57.638 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:57.632+0000 7fe37553aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:57.639 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:57.632+0000 7fe37553aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:57.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-24T17:15:57.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:15:57.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-24T17:15:57.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:15:57.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:15:57.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:57.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:15:57.832 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:15:57.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:57.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:15:58.022 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:58.016+0000 7fe37553aa40 -1 Falling back to public interface 2026-02-24T17:15:58.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:15:58.591 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:58.584+0000 7fe37553aa40 -1 osd.6 0 log_to_monitors true 2026-02-24T17:15:59.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:15:59.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:15:59.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:15:59.041 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:15:59.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:15:59.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:15:59.259 INFO:tasks.workunit.client.0.trial066.stdout:osd.6 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3412093375,v1:127.0.0.1:6851/3412093375] [v2:127.0.0.1:6852/3412093375,v1:127.0.0.1:6853/3412093375] exists,up 1208e7cb-6405-4dcb-b570-39a424e83021 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:15:59.260 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:15:59.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-24T17:15:59.262 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:15:59.263 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=69d812c7-f790-4f7a-9bcc-22f9e87610a0 2026-02-24T17:15:59.263 INFO:tasks.workunit.client.0.trial066.stdout:add osd7 69d812c7-f790-4f7a-9bcc-22f9e87610a0 2026-02-24T17:15:59.263 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 69d812c7-f790-4f7a-9bcc-22f9e87610a0' 2026-02-24T17:15:59.264 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:15:59.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDP3J1pjb4uEBAA9nJoCzXCU4prwZKvYFYnWQ== 2026-02-24T17:15:59.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDP3J1pjb4uEBAA9nJoCzXCU4prwZKvYFYnWQ=="}' 2026-02-24T17:15:59.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 69d812c7-f790-4f7a-9bcc-22f9e87610a0 -i td/test-erasure-code/7/new.json 2026-02-24T17:15:59.502 INFO:tasks.workunit.client.0.trial066.stdout:7 2026-02-24T17:15:59.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-24T17:15:59.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDP3J1pjb4uEBAA9nJoCzXCU4prwZKvYFYnWQ== --osd-uuid 69d812c7-f790-4f7a-9bcc-22f9e87610a0 2026-02-24T17:15:59.538 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:59.532+0000 7fa285eeaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:59.539 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:59.532+0000 7fa285eeaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:59.540 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:59.536+0000 7fa285eeaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:15:59.541 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:59.536+0000 7fa285eeaa40 -1 bdev(0x563fc7584400 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-24T17:15:59.541 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:15:59.536+0000 7fa285eeaa40 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-24T17:16:00.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-24T17:16:00.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:16:00.353 INFO:tasks.workunit.client.0.trial066.stdout:adding osd7 key to auth repository 2026-02-24T17:16:00.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-24T17:16:00.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:16:00.643 INFO:tasks.workunit.client.0.trial066.stdout:start osd.7 2026-02-24T17:16:00.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-24T17:16:00.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:00.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:16:00.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:16:00.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:16:00.666 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:00.660+0000 7f8225ef1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:00.667 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:00.660+0000 7f8225ef1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:00.668 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:00.660+0000 7f8225ef1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:00.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-24T17:16:00.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:00.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-24T17:16:00.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:00.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:00.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:00.861 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:16:00.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:00.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:00.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-24T17:16:01.046 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:01.040+0000 7f8225ef1a40 -1 Falling back to public interface 2026-02-24T17:16:01.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:01.635 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:01.628+0000 7f8225ef1a40 -1 osd.7 0 log_to_monitors true 2026-02-24T17:16:02.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:02.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:02.077 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:16:02.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:02.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:02.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-24T17:16:02.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:03.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:03.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:03.307 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:16:03.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:16:03.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:03.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-24T17:16:03.521 INFO:tasks.workunit.client.0.trial066.stdout:osd.7 up in weight 1 up_from 38 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/1427835101,v1:127.0.0.1:6859/1427835101] [v2:127.0.0.1:6860/1427835101,v1:127.0.0.1:6861/1427835101] exists,up 69d812c7-f790-4f7a-9bcc-22f9e87610a0 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:16:03.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:16:03.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:16:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:16:03.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:16:03.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:16:03.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:16:03.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:16:03.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:16:03.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-24T17:16:03.529 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:16:03.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=921a8c69-6a4d-4172-a04b-87df92b47bc5 2026-02-24T17:16:03.529 INFO:tasks.workunit.client.0.trial066.stdout:add osd8 921a8c69-6a4d-4172-a04b-87df92b47bc5 2026-02-24T17:16:03.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 921a8c69-6a4d-4172-a04b-87df92b47bc5' 2026-02-24T17:16:03.530 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:16:03.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDT3J1pwq7kHxAAQaEz/fLacYUbt0++Iq7wZg== 2026-02-24T17:16:03.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDT3J1pwq7kHxAAQaEz/fLacYUbt0++Iq7wZg=="}' 2026-02-24T17:16:03.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 921a8c69-6a4d-4172-a04b-87df92b47bc5 -i td/test-erasure-code/8/new.json 2026-02-24T17:16:03.757 INFO:tasks.workunit.client.0.trial066.stdout:8 2026-02-24T17:16:03.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-24T17:16:03.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDT3J1pwq7kHxAAQaEz/fLacYUbt0++Iq7wZg== --osd-uuid 921a8c69-6a4d-4172-a04b-87df92b47bc5 2026-02-24T17:16:03.793 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:03.788+0000 7ffb57e17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:03.795 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:03.788+0000 7ffb57e17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:03.796 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:03.788+0000 7ffb57e17a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:03.796 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:03.792+0000 7ffb57e17a40 -1 bdev(0x557a88b66400 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-24T17:16:03.797 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:03.792+0000 7ffb57e17a40 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-24T17:16:04.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-24T17:16:04.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:16:04.457 INFO:tasks.workunit.client.0.trial066.stdout:adding osd8 key to auth repository 2026-02-24T17:16:04.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-24T17:16:04.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:16:04.742 INFO:tasks.workunit.client.0.trial066.stdout:start osd.8 2026-02-24T17:16:04.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-24T17:16:04.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:04.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:16:04.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:16:04.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:16:04.764 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:04.756+0000 7f70744c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:04.766 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:04.760+0000 7f70744c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:04.767 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:04.760+0000 7f70744c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:04.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-24T17:16:04.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:04.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-24T17:16:04.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:04.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:04.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:04.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:04.965 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:16:04.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:04.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-24T17:16:05.170 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:05.164+0000 7f70744c8a40 -1 Falling back to public interface 2026-02-24T17:16:05.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:05.844 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:05.836+0000 7f70744c8a40 -1 osd.8 0 log_to_monitors true 2026-02-24T17:16:06.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:06.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:06.182 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:16:06.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:06.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:06.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-24T17:16:06.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:07.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:07.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:07.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:16:07.397 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:16:07.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:07.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-24T17:16:07.609 INFO:tasks.workunit.client.0.trial066.stdout:osd.8 up in weight 1 up_from 43 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/351940093,v1:127.0.0.1:6867/351940093] [v2:127.0.0.1:6868/351940093,v1:127.0.0.1:6869/351940093] exists,up 921a8c69-6a4d-4172-a04b-87df92b47bc5 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:16:07.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:16:07.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:16:07.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:16:07.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:16:07.611 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:16:07.611 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-24T17:16:07.612 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:16:07.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=85d7aa14-6d19-469a-a98e-dfd20bf36494 2026-02-24T17:16:07.613 INFO:tasks.workunit.client.0.trial066.stdout:add osd9 85d7aa14-6d19-469a-a98e-dfd20bf36494 2026-02-24T17:16:07.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 85d7aa14-6d19-469a-a98e-dfd20bf36494' 2026-02-24T17:16:07.614 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:16:07.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDX3J1plPP9JBAA/KgXXpHHdZpwjfbPyXPjzg== 2026-02-24T17:16:07.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDX3J1plPP9JBAA/KgXXpHHdZpwjfbPyXPjzg=="}' 2026-02-24T17:16:07.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 85d7aa14-6d19-469a-a98e-dfd20bf36494 -i td/test-erasure-code/9/new.json 2026-02-24T17:16:07.837 INFO:tasks.workunit.client.0.trial066.stdout:9 2026-02-24T17:16:07.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-24T17:16:07.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDX3J1plPP9JBAA/KgXXpHHdZpwjfbPyXPjzg== --osd-uuid 85d7aa14-6d19-469a-a98e-dfd20bf36494 2026-02-24T17:16:07.862 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:07.856+0000 7f46ae34ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:07.864 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:07.856+0000 7f46ae34ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:07.865 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:07.860+0000 7f46ae34ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:07.866 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:07.860+0000 7f46ae34ea40 -1 bdev(0x5620ab882400 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-24T17:16:07.866 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:07.860+0000 7f46ae34ea40 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-24T17:16:08.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-24T17:16:08.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:16:08.525 INFO:tasks.workunit.client.0.trial066.stdout:adding osd9 key to auth repository 2026-02-24T17:16:08.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-24T17:16:08.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:16:08.809 INFO:tasks.workunit.client.0.trial066.stdout:start osd.9 2026-02-24T17:16:08.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-24T17:16:08.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:08.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:16:08.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:16:08.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:16:08.832 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:08.824+0000 7f08fc268a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:08.833 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:08.828+0000 7f08fc268a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:08.835 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:08.828+0000 7f08fc268a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:09.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-24T17:16:09.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:09.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-24T17:16:09.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:09.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:09.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:09.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:09.029 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:16:09.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:09.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-24T17:16:09.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:09.246 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:09.240+0000 7f08fc268a40 -1 Falling back to public interface 2026-02-24T17:16:09.852 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:09.844+0000 7f08fc268a40 -1 osd.9 0 log_to_monitors true 2026-02-24T17:16:10.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:10.239 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:16:10.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:10.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:10.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:10.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-24T17:16:10.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:11.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:11.457 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:16:11.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:11.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:16:11.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:11.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-24T17:16:11.664 INFO:tasks.workunit.client.0.trial066.stdout:osd.9 up in weight 1 up_from 48 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/233616611,v1:127.0.0.1:6875/233616611] [v2:127.0.0.1:6876/233616611,v1:127.0.0.1:6877/233616611] exists,up 85d7aa14-6d19-469a-a98e-dfd20bf36494 2026-02-24T17:16:11.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:11.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:11.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:16:11.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:11.668 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:11.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:11.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:16:11.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:16:11.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:16:11.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:16:11.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:16:11.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:16:11.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1336efe9-d8ba-4a78-9ce4-5acfd8cb6b70 2026-02-24T17:16:11.673 INFO:tasks.workunit.client.0.trial066.stdout:add osd10 1336efe9-d8ba-4a78-9ce4-5acfd8cb6b70 2026-02-24T17:16:11.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 1336efe9-d8ba-4a78-9ce4-5acfd8cb6b70' 2026-02-24T17:16:11.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:16:11.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDb3J1pbPliKBAA5OiL3FXSp+lFtBNnI3EOgQ== 2026-02-24T17:16:11.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDb3J1pbPliKBAA5OiL3FXSp+lFtBNnI3EOgQ=="}' 2026-02-24T17:16:11.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1336efe9-d8ba-4a78-9ce4-5acfd8cb6b70 -i td/test-erasure-code/10/new.json 2026-02-24T17:16:11.905 INFO:tasks.workunit.client.0.trial066.stdout:10 2026-02-24T17:16:11.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-24T17:16:11.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDb3J1pbPliKBAA5OiL3FXSp+lFtBNnI3EOgQ== --osd-uuid 1336efe9-d8ba-4a78-9ce4-5acfd8cb6b70 2026-02-24T17:16:11.941 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:11.936+0000 7f494a4a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:11.943 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:11.936+0000 7f494a4a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:11.944 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:11.936+0000 7f494a4a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:11.944 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:11.936+0000 7f494a4a7a40 -1 bdev(0x555aa778e400 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-24T17:16:11.944 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:11.936+0000 7f494a4a7a40 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-24T17:16:12.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-24T17:16:12.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:16:12.621 INFO:tasks.workunit.client.0.trial066.stdout:adding osd10 key to auth repository 2026-02-24T17:16:12.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-24T17:16:12.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:16:12.909 INFO:tasks.workunit.client.0.trial066.stdout:start osd.10 2026-02-24T17:16:12.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-24T17:16:12.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:12.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:16:12.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:16:12.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:16:12.933 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:12.928+0000 7f1d8b4c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:12.935 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:12.928+0000 7f1d8b4c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:12.936 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:12.928+0000 7f1d8b4c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:13.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-24T17:16:13.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:13.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-24T17:16:13.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:13.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:13.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:13.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:13.126 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:16:13.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:13.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-24T17:16:13.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:13.402 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:13.396+0000 7f1d8b4c5a40 -1 Falling back to public interface 2026-02-24T17:16:14.029 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:14.024+0000 7f1d8b4c5a40 -1 osd.10 0 log_to_monitors true 2026-02-24T17:16:14.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:14.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:14.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:14.341 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:16:14.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:14.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-24T17:16:14.568 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:15.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:15.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:15.571 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:16:15.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:16:15.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:15.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-24T17:16:15.786 INFO:tasks.workunit.client.0.trial066.stdout:osd.10 up in weight 1 up_from 53 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/2637857872,v1:127.0.0.1:6883/2637857872] [v2:127.0.0.1:6884/2637857872,v1:127.0.0.1:6885/2637857872] exists,up 1336efe9-d8ba-4a78-9ce4-5acfd8cb6b70 2026-02-24T17:16:15.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:15.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:15.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:15.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-24T17:16:15.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-24T17:16:15.990 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' does not exist 2026-02-24T17:16:15.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-24T17:16:15.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:16:16.230 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:16:16.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:16:17.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-24T17:16:17.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-24T17:16:17.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:16:17.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:16:17.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:16:17.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:16:17.523 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:16:17.523 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:16:17.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:16:17.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:16:17.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:16:17.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:16:17.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:16:17.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:16:17.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:16:17.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:16:17.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:17.850 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:16:17.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836490 2026-02-24T17:16:17.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836490 2026-02-24T17:16:17.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490' 2026-02-24T17:16:17.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:17.928 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:16:18.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705674 2026-02-24T17:16:18.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705674 2026-02-24T17:16:18.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674' 2026-02-24T17:16:18.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:16:18.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574857 2026-02-24T17:16:18.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574857 2026-02-24T17:16:18.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857' 2026-02-24T17:16:18.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.091 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:16:18.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411336 2026-02-24T17:16:18.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411336 2026-02-24T17:16:18.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336' 2026-02-24T17:16:18.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:16:18.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247816 2026-02-24T17:16:18.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247816 2026-02-24T17:16:18.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336 4-98784247816' 2026-02-24T17:16:18.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.248 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:16:18.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=120259084295 2026-02-24T17:16:18.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 120259084295 2026-02-24T17:16:18.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336 4-98784247816 5-120259084295' 2026-02-24T17:16:18.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.329 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:16:18.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920774 2026-02-24T17:16:18.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920774 2026-02-24T17:16:18.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336 4-98784247816 5-120259084295 6-141733920774' 2026-02-24T17:16:18.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.407 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:16:18.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757254 2026-02-24T17:16:18.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757254 2026-02-24T17:16:18.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336 4-98784247816 5-120259084295 6-141733920774 7-163208757254' 2026-02-24T17:16:18.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.486 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:16:18.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593733 2026-02-24T17:16:18.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593733 2026-02-24T17:16:18.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336 4-98784247816 5-120259084295 6-141733920774 7-163208757254 8-184683593733' 2026-02-24T17:16:18.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.565 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:16:18.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430212 2026-02-24T17:16:18.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430212 2026-02-24T17:16:18.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336 4-98784247816 5-120259084295 6-141733920774 7-163208757254 8-184683593733 9-206158430212' 2026-02-24T17:16:18.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:18.645 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:16:18.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266691 2026-02-24T17:16:18.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266691 2026-02-24T17:16:18.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857 3-77309411336 4-98784247816 5-120259084295 6-141733920774 7-163208757254 8-184683593733 9-206158430212 10-227633266691' 2026-02-24T17:16:18.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:18.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836490 2026-02-24T17:16:18.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:18.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:16:18.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836490 2026-02-24T17:16:18.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:18.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836490 2026-02-24T17:16:18.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-24T17:16:18.726 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 21474836490 2026-02-24T17:16:18.726 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:18.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-24T17:16:18.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:16:19.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:16:19.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:20.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-24T17:16:20.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:20.152 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705674 2026-02-24T17:16:20.153 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:20.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:16:20.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705674 2026-02-24T17:16:20.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:20.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705674 2026-02-24T17:16:20.156 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 38654705674 2026-02-24T17:16:20.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-24T17:16:20.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:16:20.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-24T17:16:20.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:20.376 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574857 2026-02-24T17:16:20.376 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:20.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:16:20.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574857 2026-02-24T17:16:20.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:20.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574857 2026-02-24T17:16:20.380 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 55834574857 2026-02-24T17:16:20.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-24T17:16:20.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:16:20.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-24T17:16:20.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:20.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-77309411336 2026-02-24T17:16:20.606 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:20.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:16:20.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-77309411336 2026-02-24T17:16:20.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:20.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411336 2026-02-24T17:16:20.609 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 77309411336 2026-02-24T17:16:20.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 77309411336' 2026-02-24T17:16:20.610 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:16:20.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411337 -lt 77309411336 2026-02-24T17:16:20.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:20.835 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247816 2026-02-24T17:16:20.835 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:20.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:16:20.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247816 2026-02-24T17:16:20.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:20.838 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247816 2026-02-24T17:16:20.838 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247816' 2026-02-24T17:16:20.838 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 98784247816 2026-02-24T17:16:20.839 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:16:21.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247816 -lt 98784247816 2026-02-24T17:16:21.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:21.063 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-120259084295 2026-02-24T17:16:21.063 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:21.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:16:21.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-120259084295 2026-02-24T17:16:21.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:21.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=120259084295 2026-02-24T17:16:21.066 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 120259084295 2026-02-24T17:16:21.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 120259084295' 2026-02-24T17:16:21.067 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:16:21.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 120259084295 -lt 120259084295 2026-02-24T17:16:21.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:21.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920774 2026-02-24T17:16:21.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:21.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:16:21.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920774 2026-02-24T17:16:21.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:21.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920774 2026-02-24T17:16:21.291 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 141733920774 2026-02-24T17:16:21.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920774' 2026-02-24T17:16:21.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:16:21.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920775 -lt 141733920774 2026-02-24T17:16:21.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:21.514 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757254 2026-02-24T17:16:21.514 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:21.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:16:21.516 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757254 2026-02-24T17:16:21.516 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:21.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757254 2026-02-24T17:16:21.518 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.7 seq 163208757254 2026-02-24T17:16:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757254' 2026-02-24T17:16:21.518 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:16:21.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757254 -lt 163208757254 2026-02-24T17:16:21.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:21.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593733 2026-02-24T17:16:21.746 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:21.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:16:21.747 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593733 2026-02-24T17:16:21.748 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:21.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593733 2026-02-24T17:16:21.749 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.8 seq 184683593733 2026-02-24T17:16:21.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593733' 2026-02-24T17:16:21.750 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:16:21.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593733 -lt 184683593733 2026-02-24T17:16:21.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:21.974 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-206158430212 2026-02-24T17:16:21.974 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:21.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:16:21.976 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-206158430212 2026-02-24T17:16:21.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:21.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430212 2026-02-24T17:16:21.978 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.9 seq 206158430212 2026-02-24T17:16:21.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 206158430212' 2026-02-24T17:16:21.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:16:22.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430212 -lt 206158430212 2026-02-24T17:16:22.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:22.200 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266691 2026-02-24T17:16:22.200 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:22.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:16:22.202 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266691 2026-02-24T17:16:22.202 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:22.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266691 2026-02-24T17:16:22.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266691' 2026-02-24T17:16:22.204 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.10 seq 227633266691 2026-02-24T17:16:22.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:16:22.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266692 -lt 227633266691 2026-02-24T17:16:22.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:16:22.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:22.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:22.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-24T17:16:22.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:16:22.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:16:22.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:16:22.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:16:22.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:16:22.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:16:22.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:16:22.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-24T17:16:22.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:16:22.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:22.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:23.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-24T17:16:23.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:16:23.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.45680/ceph-osd.0.asok 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-24T17:16:23.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.45680/ceph-osd.0.asok log flush 2026-02-24T17:16:23.263 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:16:23.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-24T17:16:23.270 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:15:36.408+0000 7f7dd4c44a40 0 load: jerasure load: lrc 2026-02-24T17:16:23.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-24T17:16:23.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-24T17:16:23.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-24T17:16:23.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-24T17:16:23.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-24T17:16:23.901 INFO:tasks.workunit.client.0.trial066.stderr:pool 'ecpool' already exists 2026-02-24T17:16:23.912 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:16:24.913 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:16:24.914 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:16:24.914 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:16:24.914 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:16:24.915 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:16:24.915 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:16:24.915 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:16:24.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:16:24.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:16:24.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:16:25.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:16:25.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:16:25.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:16:25.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:16:25.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:16:25.012 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:16:25.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:16:25.222 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:25.222 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:16:25.222 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:16:25.222 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:16:25.222 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:25.223 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:25.223 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:16:25.223 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:16:25.223 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:25.223 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:16:25.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:16:25.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.224 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:16:25.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836494 2026-02-24T17:16:25.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836494 2026-02-24T17:16:25.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494' 2026-02-24T17:16:25.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.303 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:16:25.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705677 2026-02-24T17:16:25.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705677 2026-02-24T17:16:25.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677' 2026-02-24T17:16:25.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:16:25.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574860 2026-02-24T17:16:25.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574860 2026-02-24T17:16:25.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860' 2026-02-24T17:16:25.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.462 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:16:25.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411340 2026-02-24T17:16:25.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411340 2026-02-24T17:16:25.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340' 2026-02-24T17:16:25.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:16:25.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247819 2026-02-24T17:16:25.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247819 2026-02-24T17:16:25.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340 4-98784247819' 2026-02-24T17:16:25.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:16:25.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=120259084298 2026-02-24T17:16:25.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 120259084298 2026-02-24T17:16:25.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340 4-98784247819 5-120259084298' 2026-02-24T17:16:25.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.698 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:16:25.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920778 2026-02-24T17:16:25.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920778 2026-02-24T17:16:25.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340 4-98784247819 5-120259084298 6-141733920778' 2026-02-24T17:16:25.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.779 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:16:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757257 2026-02-24T17:16:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757257 2026-02-24T17:16:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340 4-98784247819 5-120259084298 6-141733920778 7-163208757257' 2026-02-24T17:16:25.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.858 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:16:25.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593736 2026-02-24T17:16:25.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593736 2026-02-24T17:16:25.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340 4-98784247819 5-120259084298 6-141733920778 7-163208757257 8-184683593736' 2026-02-24T17:16:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:25.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:16:26.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430215 2026-02-24T17:16:26.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430215 2026-02-24T17:16:26.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340 4-98784247819 5-120259084298 6-141733920778 7-163208757257 8-184683593736 9-206158430215' 2026-02-24T17:16:26.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:26.017 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:16:26.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266695 2026-02-24T17:16:26.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266695 2026-02-24T17:16:26.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836494 1-38654705677 2-55834574860 3-77309411340 4-98784247819 5-120259084298 6-141733920778 7-163208757257 8-184683593736 9-206158430215 10-227633266695' 2026-02-24T17:16:26.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:26.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836494 2026-02-24T17:16:26.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:26.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:16:26.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836494 2026-02-24T17:16:26.099 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:26.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836494 2026-02-24T17:16:26.100 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 21474836494 2026-02-24T17:16:26.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836494' 2026-02-24T17:16:26.101 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:26.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836492 -lt 21474836494 2026-02-24T17:16:26.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:16:27.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:16:27.317 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:27.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836494 -lt 21474836494 2026-02-24T17:16:27.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:27.533 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705677 2026-02-24T17:16:27.534 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:27.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:16:27.536 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705677 2026-02-24T17:16:27.536 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:27.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705677 2026-02-24T17:16:27.537 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 38654705677 2026-02-24T17:16:27.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-24T17:16:27.538 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:16:27.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705678 -lt 38654705677 2026-02-24T17:16:27.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:27.746 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574860 2026-02-24T17:16:27.747 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:27.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:16:27.749 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574860 2026-02-24T17:16:27.749 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:27.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574860 2026-02-24T17:16:27.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574860' 2026-02-24T17:16:27.750 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 55834574860 2026-02-24T17:16:27.750 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:16:27.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574861 -lt 55834574860 2026-02-24T17:16:27.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:27.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-77309411340 2026-02-24T17:16:27.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:27.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:16:27.981 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-77309411340 2026-02-24T17:16:27.981 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:27.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411340 2026-02-24T17:16:27.983 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 77309411340 2026-02-24T17:16:27.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 77309411340' 2026-02-24T17:16:27.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:16:28.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411340 -lt 77309411340 2026-02-24T17:16:28.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:28.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247819 2026-02-24T17:16:28.205 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:28.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:16:28.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247819 2026-02-24T17:16:28.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:28.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247819 2026-02-24T17:16:28.208 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 98784247819 2026-02-24T17:16:28.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247819' 2026-02-24T17:16:28.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:16:28.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247819 -lt 98784247819 2026-02-24T17:16:28.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:28.424 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-120259084298 2026-02-24T17:16:28.425 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:28.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:16:28.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-120259084298 2026-02-24T17:16:28.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:28.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=120259084298 2026-02-24T17:16:28.428 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 120259084298 2026-02-24T17:16:28.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 120259084298' 2026-02-24T17:16:28.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:16:28.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 120259084299 -lt 120259084298 2026-02-24T17:16:28.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:28.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920778 2026-02-24T17:16:28.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:28.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:16:28.660 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920778 2026-02-24T17:16:28.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:28.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920778 2026-02-24T17:16:28.662 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 141733920778 2026-02-24T17:16:28.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920778' 2026-02-24T17:16:28.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:16:28.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920778 -lt 141733920778 2026-02-24T17:16:28.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:28.883 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757257 2026-02-24T17:16:28.883 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:28.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:16:28.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757257 2026-02-24T17:16:28.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:28.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757257 2026-02-24T17:16:28.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757257' 2026-02-24T17:16:28.887 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.7 seq 163208757257 2026-02-24T17:16:28.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:16:29.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757258 -lt 163208757257 2026-02-24T17:16:29.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:29.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593736 2026-02-24T17:16:29.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:29.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:16:29.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593736 2026-02-24T17:16:29.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:29.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593736 2026-02-24T17:16:29.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593736' 2026-02-24T17:16:29.115 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.8 seq 184683593736 2026-02-24T17:16:29.115 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:16:29.333 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593737 -lt 184683593736 2026-02-24T17:16:29.334 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:29.334 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-206158430215 2026-02-24T17:16:29.335 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:29.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:16:29.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-206158430215 2026-02-24T17:16:29.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:29.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430215 2026-02-24T17:16:29.338 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.9 seq 206158430215 2026-02-24T17:16:29.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 206158430215' 2026-02-24T17:16:29.338 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:16:29.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430216 -lt 206158430215 2026-02-24T17:16:29.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:29.568 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266695 2026-02-24T17:16:29.568 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:29.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:16:29.570 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266695 2026-02-24T17:16:29.570 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:29.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266695 2026-02-24T17:16:29.572 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.10 seq 227633266695 2026-02-24T17:16:29.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266695' 2026-02-24T17:16:29.572 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:16:29.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266695 -lt 227633266695 2026-02-24T17:16:29.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:16:29.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:29.789 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:30.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-24T17:16:30.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:16:30.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:16:30.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:16:30.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:16:30.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:16:30.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:16:30.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:16:30.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-24T17:16:30.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:16:30.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:30.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:30.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-24T17:16:30.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:16:30.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:16:30.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-24T17:16:30.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T17:16:30.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-24T17:16:30.561 INFO:tasks.workunit.client.0.trial066.stderr:TEST_chunk_mapping 2026-02-24T17:16:30.561 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_put_get_isa 2026-02-24T17:16:30.561 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_put_get_jerasure 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_put_get_lrc_advanced 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_put_get_lrc_kml 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_put_get_shec' 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: local payload=ABC 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:267: TEST_alignment_constraints: echo ABC 2026-02-24T17:16:30.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: chunk_size 2026-02-24T17:16:30.563 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-24T17:16:30.571 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-24T17:16:30.572 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-24T17:16:30.572 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-24T17:16:30.586 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-24T17:16:30.587 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-24T17:16:30.587 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-24T17:16:30.587 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-24T17:16:30.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:273: TEST_alignment_constraints: local stripe_unit=4096 2026-02-24T17:16:30.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-24T17:16:30.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: grep k= 2026-02-24T17:16:30.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: eval local k=2 2026-02-24T17:16:30.810 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:274: TEST_alignment_constraints: local k=2 2026-02-24T17:16:30.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:275: TEST_alignment_constraints: local block_size=8191 2026-02-24T17:16:30.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:276: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-24T17:16:30.811 INFO:tasks.workunit.client.0.trial066.stderr:2+0 records in 2026-02-24T17:16:30.811 INFO:tasks.workunit.client.0.trial066.stderr:2+0 records out 2026-02-24T17:16:30.811 INFO:tasks.workunit.client.0.trial066.stderr:16382 bytes (16 kB, 16 KiB) copied, 0.000124466 s, 132 MB/s 2026-02-24T17:16:30.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:277: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-24T17:16:30.832 INFO:tasks.workunit.client.0.trial066.stderr:INFO: op_size has been rounded to 8192 2026-02-24T17:16:30.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:279: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:320: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=ecpool 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=0 2026-02-24T17:16:30.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=1 2026-02-24T17:16:30.840 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-24T17:16:30.840 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-24T17:16:30.850 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-24T17:16:30.850 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-24T17:16:30.851 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-24T17:16:30.867 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-24T17:16:30.867 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-24T17:16:30.867 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-24T17:16:30.867 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-24T17:16:30.867 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-24T17:16:30.868 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-24T17:16:30.868 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-24T17:16:30.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-24T17:16:30.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-24T17:16:30.879 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-24T17:16:30.892 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-24T17:16:30.892 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-24T17:16:30.892 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-24T17:16:30.892 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-24T17:16:30.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-24T17:16:30.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-24T17:16:30.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-24T17:16:30.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-24T17:16:30.913 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-24T17:16:30.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-24T17:16:30.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-24T17:16:30.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-24T17:16:30.937 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-24T17:16:30.937 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:0' 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-24T17:16:31.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-24T17:16:31.217 INFO:tasks.workunit.client.0.trial066.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-24T17:16:31.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-24T17:16:31.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-24T17:16:31.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-24T17:16:31.293 INFO:tasks.workunit.client.0.trial066.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-24T17:16:31.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-24T17:16:31.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-24T17:16:31.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-24T17:16:31.367 INFO:tasks.workunit.client.0.trial066.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-24T17:16:31.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-24T17:16:31.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-24T17:16:31.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-24T17:16:31.436 INFO:tasks.workunit.client.0.trial066.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-24T17:16:31.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-24T17:16:31.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 4 )) 2026-02-24T17:16:31.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:16:31.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-24T17:16:31.441 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-24T17:16:31.441 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-24T17:16:31.441 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-24T17:16:31.442 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-24T17:16:31.442 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:16:31.654 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr:0' 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-24T17:16:31.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:16:31.656 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:16:31.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:16:31.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:16:31.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:16:31.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:16:31.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-24T17:16:31.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-24T17:16:31.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:16:31.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-24T17:16:31.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:16:31.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-24T17:16:31.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-24T17:16:32.164 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-24T17:16:32.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-24T17:16:32.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:16:32.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T17:16:32.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:16:32.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-24T17:16:32.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:16:32.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:16:32.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:16:32.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:16:32.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-24T17:16:32.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-24T17:16:32.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:16:32.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:16:32.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:16:32.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:16:32.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:16:32.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T17:16:32.172 INFO:tasks.workunit.client.0.trial066.stderr:start osd.3 2026-02-24T17:16:32.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:32.173 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-24T17:16:32.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T17:16:32.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:16:32.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:16:32.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:16:32.184 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:32.181+0000 7f5d84c35a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:32.185 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:32.181+0000 7f5d84c35a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:32.186 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:32.181+0000 7f5d84c35a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:32.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T17:16:32.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:32.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:16:32.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:32.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:32.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:32.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:32.389 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:16:32.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:32.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:16:32.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:32.696 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:32.693+0000 7f5d84c35a40 -1 Falling back to public interface 2026-02-24T17:16:33.342 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:33.338+0000 7f5d84c35a40 -1 osd.3 74 log_to_monitors true 2026-02-24T17:16:33.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:33.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:33.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:33.599 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:33.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:33.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:16:33.809 INFO:tasks.workunit.client.0.trial066.stderr:osd.3 up in weight 1 up_from 78 up_thru 78 down_at 75 last_clean_interval [18,74) [v2:127.0.0.1:6826/3491583595,v1:127.0.0.1:6827/3491583595] [v2:127.0.0.1:6828/3491583595,v1:127.0.0.1:6829/3491583595] exists,up bdee9d30-7c32-48b2-a655-2f158e55fcf2 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:16:33.810 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:16:33.811 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:16:33.811 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:16:33.811 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:16:33.811 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:16:33.811 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:16:33.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:16:33.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:16:33.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:16:33.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:16:33.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:16:33.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:16:34.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:16:34.132 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:34.132 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:16:34.132 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:16:34.132 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.133 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:16:34.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836498 2026-02-24T17:16:34.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836498 2026-02-24T17:16:34.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498' 2026-02-24T17:16:34.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:16:34.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705681 2026-02-24T17:16:34.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705681 2026-02-24T17:16:34.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681' 2026-02-24T17:16:34.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.290 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:16:34.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574864 2026-02-24T17:16:34.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574864 2026-02-24T17:16:34.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864' 2026-02-24T17:16:34.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.367 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:16:34.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449091 2026-02-24T17:16:34.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449091 2026-02-24T17:16:34.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091' 2026-02-24T17:16:34.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.449 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:16:34.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247823 2026-02-24T17:16:34.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247823 2026-02-24T17:16:34.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091 4-98784247823' 2026-02-24T17:16:34.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.530 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:16:34.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=120259084302 2026-02-24T17:16:34.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 120259084302 2026-02-24T17:16:34.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091 4-98784247823 5-120259084302' 2026-02-24T17:16:34.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:16:34.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920781 2026-02-24T17:16:34.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920781 2026-02-24T17:16:34.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091 4-98784247823 5-120259084302 6-141733920781' 2026-02-24T17:16:34.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.688 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:16:34.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757261 2026-02-24T17:16:34.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757261 2026-02-24T17:16:34.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091 4-98784247823 5-120259084302 6-141733920781 7-163208757261' 2026-02-24T17:16:34.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.770 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:16:34.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593740 2026-02-24T17:16:34.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593740 2026-02-24T17:16:34.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091 4-98784247823 5-120259084302 6-141733920781 7-163208757261 8-184683593740' 2026-02-24T17:16:34.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.846 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:16:34.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430219 2026-02-24T17:16:34.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430219 2026-02-24T17:16:34.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091 4-98784247823 5-120259084302 6-141733920781 7-163208757261 8-184683593740 9-206158430219' 2026-02-24T17:16:34.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:34.925 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:16:35.003 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266698 2026-02-24T17:16:35.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266698 2026-02-24T17:16:35.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836498 1-38654705681 2-55834574864 3-335007449091 4-98784247823 5-120259084302 6-141733920781 7-163208757261 8-184683593740 9-206158430219 10-227633266698' 2026-02-24T17:16:35.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:35.004 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836498 2026-02-24T17:16:35.004 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:35.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:16:35.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836498 2026-02-24T17:16:35.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:35.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836498 2026-02-24T17:16:35.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836498' 2026-02-24T17:16:35.007 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 21474836498 2026-02-24T17:16:35.007 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:35.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836498 -lt 21474836498 2026-02-24T17:16:35.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:35.225 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705681 2026-02-24T17:16:35.226 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:35.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:16:35.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705681 2026-02-24T17:16:35.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:35.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705681 2026-02-24T17:16:35.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705681' 2026-02-24T17:16:35.229 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 38654705681 2026-02-24T17:16:35.229 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:16:35.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705681 -lt 38654705681 2026-02-24T17:16:35.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:35.452 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574864 2026-02-24T17:16:35.452 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:35.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:16:35.454 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574864 2026-02-24T17:16:35.454 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:35.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574864 2026-02-24T17:16:35.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574864' 2026-02-24T17:16:35.456 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 55834574864 2026-02-24T17:16:35.456 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:16:35.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574864 -lt 55834574864 2026-02-24T17:16:35.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:35.682 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-335007449091 2026-02-24T17:16:35.683 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:35.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:16:35.684 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-335007449091 2026-02-24T17:16:35.685 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:35.686 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449091 2026-02-24T17:16:35.686 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 335007449091' 2026-02-24T17:16:35.686 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 335007449091 2026-02-24T17:16:35.686 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:16:35.895 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449091 -lt 335007449091 2026-02-24T17:16:35.895 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:35.895 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247823 2026-02-24T17:16:35.895 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:35.896 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:16:35.897 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247823 2026-02-24T17:16:35.897 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:35.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247823 2026-02-24T17:16:35.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247823' 2026-02-24T17:16:35.899 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.4 seq 98784247823 2026-02-24T17:16:35.899 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:16:36.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247823 -lt 98784247823 2026-02-24T17:16:36.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:36.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-120259084302 2026-02-24T17:16:36.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:36.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:16:36.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-120259084302 2026-02-24T17:16:36.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:36.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=120259084302 2026-02-24T17:16:36.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 120259084302' 2026-02-24T17:16:36.124 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.5 seq 120259084302 2026-02-24T17:16:36.125 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:16:36.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 120259084302 -lt 120259084302 2026-02-24T17:16:36.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:36.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920781 2026-02-24T17:16:36.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:36.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:16:36.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920781 2026-02-24T17:16:36.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:36.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920781 2026-02-24T17:16:36.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920781' 2026-02-24T17:16:36.345 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.6 seq 141733920781 2026-02-24T17:16:36.346 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:16:36.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920782 -lt 141733920781 2026-02-24T17:16:36.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:36.556 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757261 2026-02-24T17:16:36.556 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:36.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:16:36.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757261 2026-02-24T17:16:36.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:36.560 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757261 2026-02-24T17:16:36.560 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757261' 2026-02-24T17:16:36.560 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.7 seq 163208757261 2026-02-24T17:16:36.560 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:16:36.785 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757261 -lt 163208757261 2026-02-24T17:16:36.785 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:36.786 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593740 2026-02-24T17:16:36.786 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:36.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:16:36.787 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593740 2026-02-24T17:16:36.787 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:36.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593740 2026-02-24T17:16:36.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593740' 2026-02-24T17:16:36.789 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.8 seq 184683593740 2026-02-24T17:16:36.789 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:16:37.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593740 -lt 184683593740 2026-02-24T17:16:37.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:37.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-206158430219 2026-02-24T17:16:37.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:37.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:16:37.014 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-206158430219 2026-02-24T17:16:37.015 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:37.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430219 2026-02-24T17:16:37.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 206158430219' 2026-02-24T17:16:37.016 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.9 seq 206158430219 2026-02-24T17:16:37.017 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:16:37.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430220 -lt 206158430219 2026-02-24T17:16:37.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:37.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266698 2026-02-24T17:16:37.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:37.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:16:37.243 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266698 2026-02-24T17:16:37.243 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:37.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266698 2026-02-24T17:16:37.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266698' 2026-02-24T17:16:37.244 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.10 seq 227633266698 2026-02-24T17:16:37.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:16:37.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266699 -lt 227633266698 2026-02-24T17:16:37.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:16:37.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:37.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:37.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-24T17:16:37.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:16:37.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:16:37.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:16:37.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:16:37.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:16:37.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:16:37.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:16:37.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-24T17:16:37.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:16:37.953 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:37.953 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:38.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-24T17:16:38.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:16:38.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:16:38.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-24T17:16:38.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-24T17:16:38.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:16:38.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-24T17:16:38.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:16:38.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:16:38.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:16:38.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:16:38.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:16:38.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-24T17:16:38.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-24T17:16:38.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:16:38.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-24T17:16:38.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:16:38.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-24T17:16:38.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-24T17:16:38.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-24T17:16:38.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-24T17:16:38.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:38.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-24T17:16:38.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-24T17:16:38.720 INFO:tasks.workunit.client.0.trial066.stderr:start osd.5 2026-02-24T17:16:38.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:38.721 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-24T17:16:38.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-24T17:16:38.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:16:38.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:16:38.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:16:38.736 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:38.735+0000 7f888fbeba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:38.737 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:38.739+0000 7f888fbeba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:38.738 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:38.739+0000 7f888fbeba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:38.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:16:39.127 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:39.127+0000 7f888fbeba40 -1 Falling back to public interface 2026-02-24T17:16:39.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:39.760 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:39.759+0000 7f888fbeba40 -1 osd.5 79 log_to_monitors true 2026-02-24T17:16:40.148 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:40.148 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:40.148 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:40.148 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:40.148 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:40.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:16:40.356 INFO:tasks.workunit.client.0.trial066.stderr:osd.5 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [28,79) [v2:127.0.0.1:6842/1349159464,v1:127.0.0.1:6843/1349159464] [v2:127.0.0.1:6844/1349159464,v1:127.0.0.1:6845/1349159464] exists,up a9ded35c-a91d-458a-aca3-7d5b2947ee4b 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:16:40.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:16:40.358 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:16:40.358 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:16:40.359 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:16:40.359 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:16:40.359 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:16:40.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:16:40.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:16:40.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:16:40.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:16:40.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:16:40.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:16:40.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:16:40.680 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:16:40.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:16:40.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:40.682 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:16:40.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836501 2026-02-24T17:16:40.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836501 2026-02-24T17:16:40.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501' 2026-02-24T17:16:40.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:40.758 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:16:40.838 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705684 2026-02-24T17:16:40.838 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705684 2026-02-24T17:16:40.838 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684' 2026-02-24T17:16:40.838 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:40.838 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:16:40.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574868 2026-02-24T17:16:40.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574868 2026-02-24T17:16:40.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868' 2026-02-24T17:16:40.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:40.918 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:16:40.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449094 2026-02-24T17:16:40.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449094 2026-02-24T17:16:40.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094' 2026-02-24T17:16:40.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:40.998 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:16:41.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247826 2026-02-24T17:16:41.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247826 2026-02-24T17:16:41.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094 4-98784247826' 2026-02-24T17:16:41.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:41.074 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:16:41.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285571 2026-02-24T17:16:41.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285571 2026-02-24T17:16:41.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094 4-98784247826 5-356482285571' 2026-02-24T17:16:41.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:41.153 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:16:41.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920785 2026-02-24T17:16:41.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920785 2026-02-24T17:16:41.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094 4-98784247826 5-356482285571 6-141733920785' 2026-02-24T17:16:41.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:41.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:16:41.312 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757264 2026-02-24T17:16:41.312 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757264 2026-02-24T17:16:41.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094 4-98784247826 5-356482285571 6-141733920785 7-163208757264' 2026-02-24T17:16:41.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:41.313 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:16:41.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593743 2026-02-24T17:16:41.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593743 2026-02-24T17:16:41.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094 4-98784247826 5-356482285571 6-141733920785 7-163208757264 8-184683593743' 2026-02-24T17:16:41.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:41.395 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:16:41.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430223 2026-02-24T17:16:41.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430223 2026-02-24T17:16:41.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094 4-98784247826 5-356482285571 6-141733920785 7-163208757264 8-184683593743 9-206158430223' 2026-02-24T17:16:41.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:41.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:16:41.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266702 2026-02-24T17:16:41.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266702 2026-02-24T17:16:41.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836501 1-38654705684 2-55834574868 3-335007449094 4-98784247826 5-356482285571 6-141733920785 7-163208757264 8-184683593743 9-206158430223 10-227633266702' 2026-02-24T17:16:41.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:41.555 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836501 2026-02-24T17:16:41.555 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:41.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:16:41.556 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836501 2026-02-24T17:16:41.557 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:41.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836501 2026-02-24T17:16:41.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836501' 2026-02-24T17:16:41.559 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 21474836501 2026-02-24T17:16:41.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:41.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836501 -lt 21474836501 2026-02-24T17:16:41.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:41.772 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705684 2026-02-24T17:16:41.773 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:41.774 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:16:41.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705684 2026-02-24T17:16:41.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:41.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705684 2026-02-24T17:16:41.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705684' 2026-02-24T17:16:41.776 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 38654705684 2026-02-24T17:16:41.776 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:16:41.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705684 -lt 38654705684 2026-02-24T17:16:41.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:41.986 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574868 2026-02-24T17:16:41.986 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:41.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:16:41.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574868 2026-02-24T17:16:41.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:41.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574868 2026-02-24T17:16:41.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574868' 2026-02-24T17:16:41.989 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 55834574868 2026-02-24T17:16:41.990 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:16:42.209 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574868 -lt 55834574868 2026-02-24T17:16:42.210 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:42.210 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-335007449094 2026-02-24T17:16:42.210 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:42.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:16:42.212 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-335007449094 2026-02-24T17:16:42.212 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:42.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449094 2026-02-24T17:16:42.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 335007449094' 2026-02-24T17:16:42.213 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 335007449094 2026-02-24T17:16:42.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:16:42.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449094 -lt 335007449094 2026-02-24T17:16:42.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:42.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247826 2026-02-24T17:16:42.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:42.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:16:42.438 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247826 2026-02-24T17:16:42.438 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:42.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247826 2026-02-24T17:16:42.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247826' 2026-02-24T17:16:42.439 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.4 seq 98784247826 2026-02-24T17:16:42.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:16:42.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247824 -lt 98784247826 2026-02-24T17:16:42.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:16:43.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:16:43.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:16:43.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247827 -lt 98784247826 2026-02-24T17:16:43.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:43.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-356482285571 2026-02-24T17:16:43.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:43.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:16:43.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-356482285571 2026-02-24T17:16:43.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:43.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285571 2026-02-24T17:16:43.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 356482285571' 2026-02-24T17:16:43.879 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.5 seq 356482285571 2026-02-24T17:16:43.880 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:16:44.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-24T17:16:44.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:44.102 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-141733920785 2026-02-24T17:16:44.102 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:44.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:16:44.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-141733920785 2026-02-24T17:16:44.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:44.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920785 2026-02-24T17:16:44.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 141733920785' 2026-02-24T17:16:44.105 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.6 seq 141733920785 2026-02-24T17:16:44.105 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:16:44.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920785 -lt 141733920785 2026-02-24T17:16:44.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:44.327 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757264 2026-02-24T17:16:44.327 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:44.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:16:44.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757264 2026-02-24T17:16:44.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:44.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757264 2026-02-24T17:16:44.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757264' 2026-02-24T17:16:44.330 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.7 seq 163208757264 2026-02-24T17:16:44.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:16:44.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757265 -lt 163208757264 2026-02-24T17:16:44.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:44.553 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593743 2026-02-24T17:16:44.553 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:44.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:16:44.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593743 2026-02-24T17:16:44.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:44.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593743 2026-02-24T17:16:44.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593743' 2026-02-24T17:16:44.556 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.8 seq 184683593743 2026-02-24T17:16:44.556 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:16:44.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593744 -lt 184683593743 2026-02-24T17:16:44.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:44.777 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-206158430223 2026-02-24T17:16:44.777 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:44.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:16:44.779 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-206158430223 2026-02-24T17:16:44.779 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:44.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430223 2026-02-24T17:16:44.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 206158430223' 2026-02-24T17:16:44.780 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.9 seq 206158430223 2026-02-24T17:16:44.781 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:16:44.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430223 -lt 206158430223 2026-02-24T17:16:44.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:44.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266702 2026-02-24T17:16:44.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:44.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:16:44.990 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266702 2026-02-24T17:16:44.990 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:44.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266702 2026-02-24T17:16:44.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266702' 2026-02-24T17:16:44.992 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.10 seq 227633266702 2026-02-24T17:16:44.992 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:16:45.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266702 -lt 227633266702 2026-02-24T17:16:45.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:16:45.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:45.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:45.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-24T17:16:45.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:16:45.502 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:16:45.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:16:45.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:16:45.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:16:45.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:16:45.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:16:45.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-24T17:16:45.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:16:45.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:45.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:46.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-24T17:16:46.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:16:46.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:16:46.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:322: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-24T17:16:46.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-24T17:16:46.507 INFO:tasks.workunit.client.0.trial066.stdout:crush-device-class= 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:crush-failure-domain=host 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:crush-num-failure-domains=0 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:crush-osds-per-failure-domain=0 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:crush-root=default 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:k=-1 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:l=-1 2026-02-24T17:16:46.508 INFO:tasks.workunit.client.0.trial066.stdout:layers=[ [ "cDD", "" ] ] 2026-02-24T17:16:46.509 INFO:tasks.workunit.client.0.trial066.stdout:m=-1 2026-02-24T17:16:46.509 INFO:tasks.workunit.client.0.trial066.stdout:mapping=_DD 2026-02-24T17:16:46.509 INFO:tasks.workunit.client.0.trial066.stdout:plugin=lrc 2026-02-24T17:16:46.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:328: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-24T17:16:46.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-24T17:16:46.864 INFO:tasks.workunit.client.0.trial066.stderr:pool 'remap-pool' already exists 2026-02-24T17:16:46.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:16:47.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:336: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-24T17:16:47.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-24T17:16:47.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local poolname=remap-pool 2026-02-24T17:16:47.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: local first=1 2026-02-24T17:16:47.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: local second=2 2026-02-24T17:16:47.878 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-24T17:16:47.879 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-24T17:16:47.889 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-24T17:16:47.889 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-24T17:16:47.889 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-24T17:16:47.904 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-24T17:16:47.904 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-24T17:16:47.904 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-24T17:16:47.904 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-24T17:16:47.904 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-24T17:16:47.904 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: chunk_size 2026-02-24T17:16:47.905 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-24T17:16:47.913 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: chunk_size: local chunk_size=0 2026-02-24T17:16:47.913 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: chunk_size: '[' 0 -eq 0 ']' 2026-02-24T17:16:47.913 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: ceph-conf --show-config-value osd_pool_default_flag_ec_optimizations 2026-02-24T17:16:47.928 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:255: chunk_size: local ec_opt=false 2026-02-24T17:16:47.928 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:256: chunk_size: '[' false = true ']' 2026-02-24T17:16:47.928 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:259: chunk_size: chunk_size=4096 2026-02-24T17:16:47.928 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: chunk_size: echo 4096 2026-02-24T17:16:47.928 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-24T17:16:47.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-24T17:16:47.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-24T17:16:47.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-24T17:16:47.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-24T17:16:47.972 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-24T17:16:47.972 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-24T17:16:47.972 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-24T17:16:47.973 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-24T17:16:47.973 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr:9' 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: local -a osds 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i = 0 )) 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-24T17:16:48.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-24T17:16:48.250 INFO:tasks.workunit.client.0.trial066.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-24T17:16:48.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-24T17:16:48.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-24T17:16:48.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-24T17:16:48.324 INFO:tasks.workunit.client.0.trial066.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-24T17:16:48.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-24T17:16:48.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-24T17:16:48.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:302: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-24T17:16:48.400 INFO:tasks.workunit.client.0.trial066.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-24T17:16:48.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i++ )) 2026-02-24T17:16:48.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:301: verify_chunk_mapping: (( i < 3 )) 2026-02-24T17:16:48.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:16:48.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:305: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-24T17:16:48.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-24T17:16:48.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-24T17:16:48.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-24T17:16:48.404 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-24T17:16:48.404 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:16:48.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-24T17:16:48.618 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:48.618 INFO:tasks.workunit.client.0.trial066.stderr:9' 2026-02-24T17:16:48.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:307: verify_chunk_mapping: local -a osds 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:308: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-24T17:16:48.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:16:48.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-24T17:16:48.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:16:48.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:16:48.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:16:48.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:16:48.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:16:48.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:16:48.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:48.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-24T17:16:48.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:16:48.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-24T17:16:48.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:16:48.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-24T17:16:48.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:49.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-24T17:16:49.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-24T17:16:49.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:16:49.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-24T17:16:49.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:16:49.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-24T17:16:49.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:16:49.139 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:16:49.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:16:49.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:16:49.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:16:49.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-24T17:16:49.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-24T17:16:49.141 INFO:tasks.workunit.client.0.trial066.stderr:start osd.6 2026-02-24T17:16:49.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:49.142 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-24T17:16:49.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-24T17:16:49.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:16:49.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:16:49.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:16:49.156 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:49.157+0000 7fbc51e05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:49.157 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:49.157+0000 7fbc51e05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:49.158 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:49.157+0000 7fbc51e05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:49.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-24T17:16:49.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:49.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-24T17:16:49.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:49.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:49.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:49.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:49.359 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:16:49.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:49.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:16:49.572 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:49.573+0000 7fbc51e05a40 -1 Falling back to public interface 2026-02-24T17:16:49.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:50.252 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:50.254+0000 7fbc51e05a40 -1 osd.6 91 log_to_monitors true 2026-02-24T17:16:50.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:50.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:50.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:50.579 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:50.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:50.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:16:50.788 INFO:tasks.workunit.client.0.trial066.stderr:osd.6 up in weight 1 up_from 95 up_thru 95 down_at 92 last_clean_interval [33,91) [v2:127.0.0.1:6850/3481951739,v1:127.0.0.1:6851/3481951739] [v2:127.0.0.1:6852/3481951739,v1:127.0.0.1:6853/3481951739] exists,up 1208e7cb-6405-4dcb-b570-39a424e83021 2026-02-24T17:16:50.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:50.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:50.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:50.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:16:50.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:16:50.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:16:50.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:16:50.789 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:16:50.790 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:16:50.790 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:16:50.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:16:50.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:16:50.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:16:50.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:16:50.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:16:50.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:16:50.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:16:50.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:16:50.898 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:16:51.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:16:51.122 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:51.122 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:16:51.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836505 2026-02-24T17:16:51.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836505 2026-02-24T17:16:51.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505' 2026-02-24T17:16:51.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.203 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:16:51.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705688 2026-02-24T17:16:51.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705688 2026-02-24T17:16:51.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688' 2026-02-24T17:16:51.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.281 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:16:51.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574872 2026-02-24T17:16:51.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574872 2026-02-24T17:16:51.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872' 2026-02-24T17:16:51.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:16:51.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449098 2026-02-24T17:16:51.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449098 2026-02-24T17:16:51.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098' 2026-02-24T17:16:51.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:16:51.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247830 2026-02-24T17:16:51.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247830 2026-02-24T17:16:51.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098 4-98784247830' 2026-02-24T17:16:51.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.516 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:16:51.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285575 2026-02-24T17:16:51.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285575 2026-02-24T17:16:51.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098 4-98784247830 5-356482285575' 2026-02-24T17:16:51.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.598 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:16:51.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=408021893123 2026-02-24T17:16:51.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 408021893123 2026-02-24T17:16:51.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098 4-98784247830 5-356482285575 6-408021893123' 2026-02-24T17:16:51.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.675 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:16:51.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757268 2026-02-24T17:16:51.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757268 2026-02-24T17:16:51.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098 4-98784247830 5-356482285575 6-408021893123 7-163208757268' 2026-02-24T17:16:51.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.754 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:16:51.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593747 2026-02-24T17:16:51.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593747 2026-02-24T17:16:51.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098 4-98784247830 5-356482285575 6-408021893123 7-163208757268 8-184683593747' 2026-02-24T17:16:51.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.833 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:16:51.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430227 2026-02-24T17:16:51.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430227 2026-02-24T17:16:51.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098 4-98784247830 5-356482285575 6-408021893123 7-163208757268 8-184683593747 9-206158430227' 2026-02-24T17:16:51.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:51.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:16:51.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266706 2026-02-24T17:16:51.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266706 2026-02-24T17:16:51.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836505 1-38654705688 2-55834574872 3-335007449098 4-98784247830 5-356482285575 6-408021893123 7-163208757268 8-184683593747 9-206158430227 10-227633266706' 2026-02-24T17:16:51.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:51.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836505 2026-02-24T17:16:51.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:51.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:16:51.989 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836505 2026-02-24T17:16:51.989 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:51.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836505 2026-02-24T17:16:51.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836505' 2026-02-24T17:16:51.991 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 21474836505 2026-02-24T17:16:51.991 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:52.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836503 -lt 21474836505 2026-02-24T17:16:52.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:16:53.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:16:53.202 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:16:53.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836505 -lt 21474836505 2026-02-24T17:16:53.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:53.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705688 2026-02-24T17:16:53.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:53.421 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:16:53.422 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705688 2026-02-24T17:16:53.422 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:53.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705688 2026-02-24T17:16:53.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705688' 2026-02-24T17:16:53.424 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 38654705688 2026-02-24T17:16:53.425 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:16:53.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705689 -lt 38654705688 2026-02-24T17:16:53.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:53.637 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574872 2026-02-24T17:16:53.637 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:53.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:16:53.639 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574872 2026-02-24T17:16:53.639 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:53.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574872 2026-02-24T17:16:53.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574872' 2026-02-24T17:16:53.640 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 55834574872 2026-02-24T17:16:53.640 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:16:53.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574872 -lt 55834574872 2026-02-24T17:16:53.853 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:53.853 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-335007449098 2026-02-24T17:16:53.853 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:53.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:16:53.855 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-335007449098 2026-02-24T17:16:53.855 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:53.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449098 2026-02-24T17:16:53.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 335007449098' 2026-02-24T17:16:53.856 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 335007449098 2026-02-24T17:16:53.857 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:16:54.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449098 -lt 335007449098 2026-02-24T17:16:54.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:54.077 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247830 2026-02-24T17:16:54.077 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:54.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:16:54.078 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247830 2026-02-24T17:16:54.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:54.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247830 2026-02-24T17:16:54.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247830' 2026-02-24T17:16:54.081 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.4 seq 98784247830 2026-02-24T17:16:54.081 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:16:54.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247831 -lt 98784247830 2026-02-24T17:16:54.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:54.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-356482285575 2026-02-24T17:16:54.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:54.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:16:54.310 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-356482285575 2026-02-24T17:16:54.310 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:54.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285575 2026-02-24T17:16:54.312 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 356482285575' 2026-02-24T17:16:54.312 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.5 seq 356482285575 2026-02-24T17:16:54.312 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:16:54.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285575 -lt 356482285575 2026-02-24T17:16:54.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:54.533 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-408021893123 2026-02-24T17:16:54.533 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:54.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:16:54.535 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-408021893123 2026-02-24T17:16:54.535 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:54.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=408021893123 2026-02-24T17:16:54.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 408021893123' 2026-02-24T17:16:54.536 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.6 seq 408021893123 2026-02-24T17:16:54.537 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:16:54.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-24T17:16:54.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:54.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757268 2026-02-24T17:16:54.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:54.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:16:54.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757268 2026-02-24T17:16:54.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:54.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757268 2026-02-24T17:16:54.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757268' 2026-02-24T17:16:54.765 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.7 seq 163208757268 2026-02-24T17:16:54.766 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:16:54.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757269 -lt 163208757268 2026-02-24T17:16:54.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:54.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593747 2026-02-24T17:16:54.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:54.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:16:54.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593747 2026-02-24T17:16:54.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:54.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593747 2026-02-24T17:16:54.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593747' 2026-02-24T17:16:54.999 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.8 seq 184683593747 2026-02-24T17:16:54.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:16:55.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593748 -lt 184683593747 2026-02-24T17:16:55.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:55.222 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-206158430227 2026-02-24T17:16:55.222 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:55.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:16:55.223 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-206158430227 2026-02-24T17:16:55.223 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:55.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430227 2026-02-24T17:16:55.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 206158430227' 2026-02-24T17:16:55.225 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.9 seq 206158430227 2026-02-24T17:16:55.225 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:16:55.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430227 -lt 206158430227 2026-02-24T17:16:55.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:55.445 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266706 2026-02-24T17:16:55.445 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:55.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:16:55.446 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266706 2026-02-24T17:16:55.446 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:55.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266706 2026-02-24T17:16:55.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266706' 2026-02-24T17:16:55.448 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.10 seq 227633266706 2026-02-24T17:16:55.448 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:16:55.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266706 -lt 227633266706 2026-02-24T17:16:55.665 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:16:55.665 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:55.665 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:55.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-24T17:16:55.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:16:55.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:16:55.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:16:55.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:16:55.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:16:55.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:16:55.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:16:56.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-24T17:16:56.162 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:16:56.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:16:56.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:16:56.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-24T17:16:56.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:16:56.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:16:56.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:56.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:309: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-24T17:16:56.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-24T17:16:56.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:16:56.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:16:56.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:16:56.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:16:56.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:16:56.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:16:56.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:56.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-24T17:16:56.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:16:56.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-24T17:16:56.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:16:56.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-24T17:16:56.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-24T17:16:56.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-24T17:16:56.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:16:56.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:16:56.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-24T17:16:56.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-24T17:16:56.960 INFO:tasks.workunit.client.0.trial066.stderr:start osd.9 2026-02-24T17:16:56.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=d8e3a5b8-324d-4201-9fe5-9c6723ebaa4b --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.45680/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:16:56.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-24T17:16:56.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-24T17:16:56.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:16:56.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:16:56.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:16:56.978 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:56.983+0000 7fbce1821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:56.980 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:56.983+0000 7fbce1821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:56.980 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:56.983+0000 7fbce1821a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:16:57.183 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:16:57.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:57.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-24T17:16:57.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:16:57.463 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:57.467+0000 7fbce1821a40 -1 Falling back to public interface 2026-02-24T17:16:58.200 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:16:58.203+0000 7fbce1821a40 -1 osd.9 98 log_to_monitors true 2026-02-24T17:16:58.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:16:58.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:16:58.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:16:58.396 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:58.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:16:58.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-24T17:16:58.603 INFO:tasks.workunit.client.0.trial066.stderr:osd.9 up in weight 1 up_from 101 up_thru 101 down_at 98 last_clean_interval [48,98) [v2:127.0.0.1:6874/81974926,v1:127.0.0.1:6875/81974926] [v2:127.0.0.1:6876/81974926,v1:127.0.0.1:6877/81974926] exists,up 85d7aa14-6d19-469a-a98e-dfd20bf36494 2026-02-24T17:16:58.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:16:58.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:16:58.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:16:58.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:16:58.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:16:58.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:16:58.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:16:58.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:16:58.605 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:16:58.605 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:16:58.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:16:58.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:16:58.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:16:58.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:16:58.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:16:58.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:16:58.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:16:58.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:16:58.700 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:58.921 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:16:59.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836509 2026-02-24T17:16:59.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836509 2026-02-24T17:16:59.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509' 2026-02-24T17:16:59.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:16:59.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705692 2026-02-24T17:16:59.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705692 2026-02-24T17:16:59.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692' 2026-02-24T17:16:59.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:16:59.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574875 2026-02-24T17:16:59.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574875 2026-02-24T17:16:59.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875' 2026-02-24T17:16:59.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.158 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:16:59.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449102 2026-02-24T17:16:59.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449102 2026-02-24T17:16:59.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102' 2026-02-24T17:16:59.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.237 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:16:59.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247834 2026-02-24T17:16:59.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247834 2026-02-24T17:16:59.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102 4-98784247834' 2026-02-24T17:16:59.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.315 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:16:59.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285578 2026-02-24T17:16:59.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285578 2026-02-24T17:16:59.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102 4-98784247834 5-356482285578' 2026-02-24T17:16:59.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:16:59.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=408021893126 2026-02-24T17:16:59.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 408021893126 2026-02-24T17:16:59.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102 4-98784247834 5-356482285578 6-408021893126' 2026-02-24T17:16:59.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.476 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:16:59.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757272 2026-02-24T17:16:59.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757272 2026-02-24T17:16:59.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102 4-98784247834 5-356482285578 6-408021893126 7-163208757272' 2026-02-24T17:16:59.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:16:59.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593751 2026-02-24T17:16:59.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593751 2026-02-24T17:16:59.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102 4-98784247834 5-356482285578 6-408021893126 7-163208757272 8-184683593751' 2026-02-24T17:16:59.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.634 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:16:59.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=433791696899 2026-02-24T17:16:59.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 433791696899 2026-02-24T17:16:59.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102 4-98784247834 5-356482285578 6-408021893126 7-163208757272 8-184683593751 9-433791696899' 2026-02-24T17:16:59.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:16:59.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:16:59.791 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266709 2026-02-24T17:16:59.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266709 2026-02-24T17:16:59.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836509 1-38654705692 2-55834574875 3-335007449102 4-98784247834 5-356482285578 6-408021893126 7-163208757272 8-184683593751 9-433791696899 10-227633266709' 2026-02-24T17:16:59.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:16:59.792 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836509 2026-02-24T17:16:59.792 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:16:59.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:16:59.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836509 2026-02-24T17:16:59.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:16:59.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836509 2026-02-24T17:16:59.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836509' 2026-02-24T17:16:59.794 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 21474836509 2026-02-24T17:16:59.794 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:17:00.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836509 -lt 21474836509 2026-02-24T17:17:00.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:00.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705692 2026-02-24T17:17:00.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:00.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:17:00.006 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705692 2026-02-24T17:17:00.006 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:00.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705692 2026-02-24T17:17:00.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705692' 2026-02-24T17:17:00.008 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 38654705692 2026-02-24T17:17:00.008 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:17:00.215 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705690 -lt 38654705692 2026-02-24T17:17:00.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:17:01.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:17:01.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:17:01.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705692 -lt 38654705692 2026-02-24T17:17:01.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:01.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574875 2026-02-24T17:17:01.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:01.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:17:01.438 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574875 2026-02-24T17:17:01.438 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:01.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574875 2026-02-24T17:17:01.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574875' 2026-02-24T17:17:01.440 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 55834574875 2026-02-24T17:17:01.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:17:01.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574876 -lt 55834574875 2026-02-24T17:17:01.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:01.667 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-335007449102 2026-02-24T17:17:01.667 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:01.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:17:01.668 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-335007449102 2026-02-24T17:17:01.668 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:01.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449102 2026-02-24T17:17:01.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 335007449102' 2026-02-24T17:17:01.670 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 335007449102 2026-02-24T17:17:01.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:17:01.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449102 -lt 335007449102 2026-02-24T17:17:01.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:01.891 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247834 2026-02-24T17:17:01.891 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:01.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:17:01.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247834 2026-02-24T17:17:01.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:01.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247834 2026-02-24T17:17:01.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247834' 2026-02-24T17:17:01.894 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.4 seq 98784247834 2026-02-24T17:17:01.894 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:17:02.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247834 -lt 98784247834 2026-02-24T17:17:02.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:02.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-356482285578 2026-02-24T17:17:02.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:02.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:17:02.125 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-356482285578 2026-02-24T17:17:02.126 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:02.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285578 2026-02-24T17:17:02.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 356482285578' 2026-02-24T17:17:02.127 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.5 seq 356482285578 2026-02-24T17:17:02.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:17:02.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285579 -lt 356482285578 2026-02-24T17:17:02.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:02.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-408021893126 2026-02-24T17:17:02.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:02.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:17:02.346 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-408021893126 2026-02-24T17:17:02.346 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:02.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=408021893126 2026-02-24T17:17:02.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 408021893126' 2026-02-24T17:17:02.349 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.6 seq 408021893126 2026-02-24T17:17:02.349 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:17:02.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 408021893127 -lt 408021893126 2026-02-24T17:17:02.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:02.578 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757272 2026-02-24T17:17:02.578 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:02.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:17:02.579 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757272 2026-02-24T17:17:02.580 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:02.581 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757272 2026-02-24T17:17:02.581 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757272' 2026-02-24T17:17:02.581 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.7 seq 163208757272 2026-02-24T17:17:02.582 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:17:02.791 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757272 -lt 163208757272 2026-02-24T17:17:02.791 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:02.792 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593751 2026-02-24T17:17:02.792 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:02.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:17:02.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593751 2026-02-24T17:17:02.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:02.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593751 2026-02-24T17:17:02.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593751' 2026-02-24T17:17:02.795 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.8 seq 184683593751 2026-02-24T17:17:02.795 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:17:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593751 -lt 184683593751 2026-02-24T17:17:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:03.018 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-433791696899 2026-02-24T17:17:03.019 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:03.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:17:03.020 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-433791696899 2026-02-24T17:17:03.020 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:03.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=433791696899 2026-02-24T17:17:03.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 433791696899' 2026-02-24T17:17:03.022 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.9 seq 433791696899 2026-02-24T17:17:03.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:17:03.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 433791696899 -lt 433791696899 2026-02-24T17:17:03.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:03.243 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266709 2026-02-24T17:17:03.243 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:03.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:17:03.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266709 2026-02-24T17:17:03.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:03.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266709 2026-02-24T17:17:03.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266709' 2026-02-24T17:17:03.246 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.10 seq 227633266709 2026-02-24T17:17:03.246 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:17:03.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266710 -lt 227633266709 2026-02-24T17:17:03.474 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:17:03.474 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:03.474 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:03.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-24T17:17:03.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:17:03.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:17:03.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:17:03.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:17:03.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:17:03.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:17:03.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:17:03.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-24T17:17:03.976 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:17:03.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:03.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:04.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-24T17:17:04.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:17:04.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:17:04.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:338: TEST_chunk_mapping: delete_pool remap-pool 2026-02-24T17:17:04.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-24T17:17:04.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-24T17:17:04.526 INFO:tasks.workunit.client.0.trial066.stderr:pool 'remap-pool' does not exist 2026-02-24T17:17:04.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:339: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-24T17:17:04.794 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile remap-profile does not exist 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: erasure_code_plugin_exists: local plugin=isa 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: erasure_code_plugin_exists: local status 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: erasure_code_plugin_exists: local grepstr 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: erasure_code_plugin_exists: local s 2026-02-24T17:17:04.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: erasure_code_plugin_exists: case `uname` in 2026-02-24T17:17:04.804 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2137: erasure_code_plugin_exists: uname 2026-02-24T17:17:04.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2139: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-24T17:17:04.805 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-24T17:17:05.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2142: erasure_code_plugin_exists: s= 2026-02-24T17:17:05.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2143: erasure_code_plugin_exists: local status=0 2026-02-24T17:17:05.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2144: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-24T17:17:05.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2145: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-24T17:17:05.352 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-24T17:17:05.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2154: erasure_code_plugin_exists: return 0 2026-02-24T17:17:05.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-24T17:17:05.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-24T17:17:05.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-24T17:17:05.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-24T17:17:05.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-24T17:17:06.048 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-isa' already exists 2026-02-24T17:17:06.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:07.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-24T17:17:07.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:07.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-24T17:17:07.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:07.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-24T17:17:07.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-24T17:17:07.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:07.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:07.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-24T17:17:07.110 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-24T17:17:07.110 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-24T17:17:07.110 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:07.110 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-24T17:17:07.110 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:07.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-24T17:17:07.332 INFO:tasks.workunit.client.0.trial066.stderr:10 2026-02-24T17:17:07.333 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:17:07.333 INFO:tasks.workunit.client.0.trial066.stderr:7' 2026-02-24T17:17:07.333 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-24T17:17:07.333 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-24T17:17:07.333 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-24T17:17:07.333 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-24T17:17:07.333 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-24T17:17:07.577 INFO:tasks.workunit.client.0.trial066.stderr:osd.7 is already out. 2026-02-24T17:17:07.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-24T17:17:12.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-24T17:17:12.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-24T17:17:12.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:12.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-24T17:17:12.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-24T17:17:12.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:12.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-24T17:17:12.803 INFO:tasks.workunit.client.0.trial066.stderr:10 2026-02-24T17:17:12.803 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:17:12.803 INFO:tasks.workunit.client.0.trial066.stderr:1' 2026-02-24T17:17:12.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-24T17:17:12.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:12.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:12.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-24T17:17:13.050 INFO:tasks.workunit.client.0.trial066.stderr:osd.7 is already in. 2026-02-24T17:17:13.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:17:13.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-24T17:17:13.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-24T17:17:13.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-24T17:17:13.352 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-isa' does not exist 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-24T17:17:13.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:13.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-24T17:17:13.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:13.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-24T17:17:13.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:13.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-24T17:17:13.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-24T17:17:13.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:13.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:13.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-24T17:17:13.413 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-24T17:17:13.413 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-24T17:17:13.413 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:13.413 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-24T17:17:13.413 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr:9' 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-24T17:17:13.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-24T17:17:13.874 INFO:tasks.workunit.client.0.trial066.stderr:osd.9 is already out. 2026-02-24T17:17:13.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-24T17:17:18.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-24T17:17:18.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-24T17:17:18.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:18.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-24T17:17:18.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-24T17:17:18.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:19.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-24T17:17:19.107 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:17:19.107 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:17:19.107 INFO:tasks.workunit.client.0.trial066.stderr:5' 2026-02-24T17:17:19.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-24T17:17:19.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:19.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:19.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-24T17:17:19.353 INFO:tasks.workunit.client.0.trial066.stderr:osd.9 is already in. 2026-02-24T17:17:19.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:17:19.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-24T17:17:19.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-24T17:17:19.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-24T17:17:19.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-24T17:17:19.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-24T17:17:20.057 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:17:20.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:17:21.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-24T17:17:21.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-24T17:17:21.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-24T17:17:21.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-24T17:17:21.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-24T17:17:21.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-24T17:17:21.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:21.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:21.116 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-24T17:17:21.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-24T17:17:21.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:17:21.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:21.119 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-24T17:17:21.119 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:21.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-24T17:17:21.330 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:17:21.330 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:21.330 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:21.330 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:21.330 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:21.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-24T17:17:21.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-24T17:17:21.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-24T17:17:21.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-24T17:17:21.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-24T17:17:21.579 INFO:tasks.workunit.client.0.trial066.stderr:osd.10 is already out. 2026-02-24T17:17:21.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-24T17:17:26.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-24T17:17:26.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:17:26.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:26.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-24T17:17:26.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-24T17:17:26.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:26.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-24T17:17:26.810 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:17:26.810 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:26.810 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:26.810 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:26.810 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:17:26.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-24T17:17:26.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:26.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:26.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-24T17:17:27.059 INFO:tasks.workunit.client.0.trial066.stderr:osd.10 is already in. 2026-02-24T17:17:27.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:17:27.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-24T17:17:27.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-24T17:17:27.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-24T17:17:27.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-24T17:17:27.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-24T17:17:27.072 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-24T17:17:27.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-24T17:17:27.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-24T17:17:27.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-24T17:17:27.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-24T17:17:27.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-24T17:17:27.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-24T17:17:27.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-24T17:17:27.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:27.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:27.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-24T17:17:27.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-24T17:17:27.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:17:27.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:17:27.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:17:27.237 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:17:27.238 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:17:27.238 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:17:27.238 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:17:27.238 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:17:27.238 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:17:27.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:17:27.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:17:27.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:17:27.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:17:27.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:17:27.348 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:17:27.560 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:27.561 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:27.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:17:27.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:27.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:17:27.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836516 2026-02-24T17:17:27.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836516 2026-02-24T17:17:27.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516' 2026-02-24T17:17:27.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:27.639 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:17:27.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705700 2026-02-24T17:17:27.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705700 2026-02-24T17:17:27.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700' 2026-02-24T17:17:27.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:27.720 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:17:27.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574883 2026-02-24T17:17:27.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574883 2026-02-24T17:17:27.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883' 2026-02-24T17:17:27.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:27.796 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:17:27.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449109 2026-02-24T17:17:27.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449109 2026-02-24T17:17:27.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109' 2026-02-24T17:17:27.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:27.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:17:27.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247842 2026-02-24T17:17:27.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247842 2026-02-24T17:17:27.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109 4-98784247842' 2026-02-24T17:17:27.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:27.953 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:17:28.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285586 2026-02-24T17:17:28.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285586 2026-02-24T17:17:28.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109 4-98784247842 5-356482285586' 2026-02-24T17:17:28.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:28.031 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:17:28.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=408021893134 2026-02-24T17:17:28.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 408021893134 2026-02-24T17:17:28.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109 4-98784247842 5-356482285586 6-408021893134' 2026-02-24T17:17:28.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:28.108 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:17:28.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757280 2026-02-24T17:17:28.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757280 2026-02-24T17:17:28.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109 4-98784247842 5-356482285586 6-408021893134 7-163208757280' 2026-02-24T17:17:28.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:28.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:17:28.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593759 2026-02-24T17:17:28.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593759 2026-02-24T17:17:28.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109 4-98784247842 5-356482285586 6-408021893134 7-163208757280 8-184683593759' 2026-02-24T17:17:28.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:28.267 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:17:28.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=433791696907 2026-02-24T17:17:28.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 433791696907 2026-02-24T17:17:28.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109 4-98784247842 5-356482285586 6-408021893134 7-163208757280 8-184683593759 9-433791696907' 2026-02-24T17:17:28.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:28.347 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:17:28.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266717 2026-02-24T17:17:28.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266717 2026-02-24T17:17:28.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836516 1-38654705700 2-55834574883 3-335007449109 4-98784247842 5-356482285586 6-408021893134 7-163208757280 8-184683593759 9-433791696907 10-227633266717' 2026-02-24T17:17:28.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:28.429 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836516 2026-02-24T17:17:28.429 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:28.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:17:28.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836516 2026-02-24T17:17:28.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:28.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836516 2026-02-24T17:17:28.432 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 21474836516 2026-02-24T17:17:28.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836516' 2026-02-24T17:17:28.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:17:28.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836514 -lt 21474836516 2026-02-24T17:17:28.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:17:29.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:17:29.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:17:29.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836517 -lt 21474836516 2026-02-24T17:17:29.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:29.860 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705700 2026-02-24T17:17:29.860 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:29.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:17:29.862 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705700 2026-02-24T17:17:29.862 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:29.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705700 2026-02-24T17:17:29.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705700' 2026-02-24T17:17:29.864 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 38654705700 2026-02-24T17:17:29.864 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:17:30.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705700 -lt 38654705700 2026-02-24T17:17:30.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:30.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574883 2026-02-24T17:17:30.077 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:30.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:17:30.078 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574883 2026-02-24T17:17:30.078 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:30.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574883 2026-02-24T17:17:30.080 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 55834574883 2026-02-24T17:17:30.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574883' 2026-02-24T17:17:30.080 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:17:30.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574883 -lt 55834574883 2026-02-24T17:17:30.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:30.307 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-335007449109 2026-02-24T17:17:30.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:30.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:17:30.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-335007449109 2026-02-24T17:17:30.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:30.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449109 2026-02-24T17:17:30.311 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 335007449109 2026-02-24T17:17:30.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 335007449109' 2026-02-24T17:17:30.312 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:17:30.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449110 -lt 335007449109 2026-02-24T17:17:30.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:30.531 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247842 2026-02-24T17:17:30.531 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:30.532 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:17:30.533 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247842 2026-02-24T17:17:30.533 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:30.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247842 2026-02-24T17:17:30.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247842' 2026-02-24T17:17:30.535 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 98784247842 2026-02-24T17:17:30.535 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:17:30.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247842 -lt 98784247842 2026-02-24T17:17:30.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:30.753 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-356482285586 2026-02-24T17:17:30.754 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:30.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:17:30.755 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-356482285586 2026-02-24T17:17:30.756 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:30.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285586 2026-02-24T17:17:30.757 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 356482285586 2026-02-24T17:17:30.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 356482285586' 2026-02-24T17:17:30.758 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:17:30.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285586 -lt 356482285586 2026-02-24T17:17:30.974 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:30.974 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-408021893134 2026-02-24T17:17:30.975 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:30.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:17:30.976 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-408021893134 2026-02-24T17:17:30.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:30.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=408021893134 2026-02-24T17:17:30.978 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 408021893134 2026-02-24T17:17:30.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 408021893134' 2026-02-24T17:17:30.978 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:17:31.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 408021893135 -lt 408021893134 2026-02-24T17:17:31.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:31.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757280 2026-02-24T17:17:31.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:31.209 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:17:31.210 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757280 2026-02-24T17:17:31.210 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:31.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757280 2026-02-24T17:17:31.212 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.7 seq 163208757280 2026-02-24T17:17:31.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757280' 2026-02-24T17:17:31.212 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:17:31.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757280 -lt 163208757280 2026-02-24T17:17:31.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:31.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593759 2026-02-24T17:17:31.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:31.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:17:31.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593759 2026-02-24T17:17:31.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:31.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593759 2026-02-24T17:17:31.435 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.8 seq 184683593759 2026-02-24T17:17:31.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593759' 2026-02-24T17:17:31.435 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:17:31.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593759 -lt 184683593759 2026-02-24T17:17:31.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:31.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-433791696907 2026-02-24T17:17:31.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:31.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:17:31.655 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-433791696907 2026-02-24T17:17:31.656 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:31.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=433791696907 2026-02-24T17:17:31.657 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.9 seq 433791696907 2026-02-24T17:17:31.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 433791696907' 2026-02-24T17:17:31.657 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:17:31.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 433791696907 -lt 433791696907 2026-02-24T17:17:31.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:31.880 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266717 2026-02-24T17:17:31.880 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:31.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:17:31.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266717 2026-02-24T17:17:31.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:31.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266717 2026-02-24T17:17:31.884 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.10 seq 227633266717 2026-02-24T17:17:31.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266717' 2026-02-24T17:17:31.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:17:32.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266718 -lt 227633266717 2026-02-24T17:17:32.099 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:17:32.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:32.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:32.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-24T17:17:32.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:17:32.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:17:32.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:17:32.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:17:32.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:17:32.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:17:32.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:17:32.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-24T17:17:32.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:17:32.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:32.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:32.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-24T17:17:32.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:17:32.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:17:32.865 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-24T17:17:32.865 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:17:32.865 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:32.866 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-24T17:17:32.866 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-24T17:17:33.087 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-24T17:17:33.331 INFO:tasks.workunit.client.0.trial066.stderr:osd.5 is already out. 2026-02-24T17:17:33.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-24T17:17:33.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-24T17:17:33.638 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 is already out. 2026-02-24T17:17:33.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-24T17:17:33.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:17:33.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:17:33.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:17:33.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:17:33.650 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:17:33.650 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:17:33.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:17:33.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:17:33.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:17:33.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:17:33.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:17:33.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:17:33.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:17:33.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:17:33.758 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:17:33.967 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:17:33.968 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:17:33.968 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:33.968 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:33.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:17:33.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:33.969 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:17:34.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836520 2026-02-24T17:17:34.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836520 2026-02-24T17:17:34.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520' 2026-02-24T17:17:34.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.050 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:17:34.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705703 2026-02-24T17:17:34.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705703 2026-02-24T17:17:34.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703' 2026-02-24T17:17:34.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.131 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:17:34.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574886 2026-02-24T17:17:34.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574886 2026-02-24T17:17:34.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886' 2026-02-24T17:17:34.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:17:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449113 2026-02-24T17:17:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449113 2026-02-24T17:17:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113' 2026-02-24T17:17:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:17:34.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247845 2026-02-24T17:17:34.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247845 2026-02-24T17:17:34.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113 4-98784247845' 2026-02-24T17:17:34.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.371 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:17:34.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285589 2026-02-24T17:17:34.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285589 2026-02-24T17:17:34.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113 4-98784247845 5-356482285589' 2026-02-24T17:17:34.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.449 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:17:34.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=408021893137 2026-02-24T17:17:34.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 408021893137 2026-02-24T17:17:34.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113 4-98784247845 5-356482285589 6-408021893137' 2026-02-24T17:17:34.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.528 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:17:34.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757283 2026-02-24T17:17:34.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757283 2026-02-24T17:17:34.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113 4-98784247845 5-356482285589 6-408021893137 7-163208757283' 2026-02-24T17:17:34.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:17:34.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593762 2026-02-24T17:17:34.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593762 2026-02-24T17:17:34.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113 4-98784247845 5-356482285589 6-408021893137 7-163208757283 8-184683593762' 2026-02-24T17:17:34.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.685 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:17:34.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=433791696910 2026-02-24T17:17:34.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 433791696910 2026-02-24T17:17:34.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113 4-98784247845 5-356482285589 6-408021893137 7-163208757283 8-184683593762 9-433791696910' 2026-02-24T17:17:34.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:34.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:17:34.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266720 2026-02-24T17:17:34.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266720 2026-02-24T17:17:34.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836520 1-38654705703 2-55834574886 3-335007449113 4-98784247845 5-356482285589 6-408021893137 7-163208757283 8-184683593762 9-433791696910 10-227633266720' 2026-02-24T17:17:34.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:34.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836520 2026-02-24T17:17:34.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:34.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:17:34.843 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836520 2026-02-24T17:17:34.843 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:34.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836520 2026-02-24T17:17:34.845 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 21474836520 2026-02-24T17:17:34.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836520' 2026-02-24T17:17:34.845 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:17:35.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836517 -lt 21474836520 2026-02-24T17:17:35.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:17:36.058 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:17:36.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:17:36.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836520 -lt 21474836520 2026-02-24T17:17:36.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:36.277 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705703 2026-02-24T17:17:36.277 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:36.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:17:36.279 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705703 2026-02-24T17:17:36.279 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:36.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705703 2026-02-24T17:17:36.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705703' 2026-02-24T17:17:36.281 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 38654705703 2026-02-24T17:17:36.281 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:17:36.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705703 -lt 38654705703 2026-02-24T17:17:36.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:36.508 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574886 2026-02-24T17:17:36.508 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:36.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:17:36.510 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574886 2026-02-24T17:17:36.510 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:36.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574886 2026-02-24T17:17:36.512 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 55834574886 2026-02-24T17:17:36.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574886' 2026-02-24T17:17:36.512 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:17:36.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574886 -lt 55834574886 2026-02-24T17:17:36.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:36.730 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-335007449113 2026-02-24T17:17:36.730 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:36.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:17:36.732 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-335007449113 2026-02-24T17:17:36.732 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:36.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449113 2026-02-24T17:17:36.733 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 335007449113 2026-02-24T17:17:36.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 335007449113' 2026-02-24T17:17:36.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:17:36.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449113 -lt 335007449113 2026-02-24T17:17:36.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:36.959 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247845 2026-02-24T17:17:36.959 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:36.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:17:36.961 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247845 2026-02-24T17:17:36.962 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:36.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247845 2026-02-24T17:17:36.963 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 98784247845 2026-02-24T17:17:36.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247845' 2026-02-24T17:17:36.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:17:37.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247845 -lt 98784247845 2026-02-24T17:17:37.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:37.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-356482285589 2026-02-24T17:17:37.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:37.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:17:37.187 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-356482285589 2026-02-24T17:17:37.187 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:37.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285589 2026-02-24T17:17:37.188 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 356482285589 2026-02-24T17:17:37.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 356482285589' 2026-02-24T17:17:37.189 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:17:37.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285590 -lt 356482285589 2026-02-24T17:17:37.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:37.408 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-408021893137 2026-02-24T17:17:37.408 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:37.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:17:37.410 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-408021893137 2026-02-24T17:17:37.410 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:37.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=408021893137 2026-02-24T17:17:37.412 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 408021893137 2026-02-24T17:17:37.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 408021893137' 2026-02-24T17:17:37.413 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:17:37.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 408021893138 -lt 408021893137 2026-02-24T17:17:37.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:37.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757283 2026-02-24T17:17:37.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:37.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:17:37.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757283 2026-02-24T17:17:37.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:37.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757283 2026-02-24T17:17:37.626 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.7 seq 163208757283 2026-02-24T17:17:37.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757283' 2026-02-24T17:17:37.627 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:17:37.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757283 -lt 163208757283 2026-02-24T17:17:37.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:37.849 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593762 2026-02-24T17:17:37.849 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:37.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:17:37.851 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593762 2026-02-24T17:17:37.851 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:37.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593762 2026-02-24T17:17:37.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593762' 2026-02-24T17:17:37.852 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.8 seq 184683593762 2026-02-24T17:17:37.853 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:17:38.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593763 -lt 184683593762 2026-02-24T17:17:38.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:38.075 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-433791696910 2026-02-24T17:17:38.075 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:38.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:17:38.077 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-433791696910 2026-02-24T17:17:38.077 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:38.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=433791696910 2026-02-24T17:17:38.078 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.9 seq 433791696910 2026-02-24T17:17:38.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 433791696910' 2026-02-24T17:17:38.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:17:38.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 433791696910 -lt 433791696910 2026-02-24T17:17:38.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:38.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266720 2026-02-24T17:17:38.288 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:38.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:17:38.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266720 2026-02-24T17:17:38.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:38.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266720 2026-02-24T17:17:38.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266720' 2026-02-24T17:17:38.291 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.10 seq 227633266720 2026-02-24T17:17:38.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:17:38.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266721 -lt 227633266720 2026-02-24T17:17:38.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:17:38.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:38.516 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:38.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-24T17:17:38.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:17:38.808 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:17:38.808 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:17:38.808 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:17:38.808 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:17:38.808 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:17:38.808 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:17:39.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-24T17:17:39.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:17:39.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:39.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:39.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-24T17:17:39.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:17:39.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:17:39.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-24T17:17:39.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-24T17:17:39.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:17:39.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:39.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-24T17:17:39.320 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-24T17:17:39.320 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:39.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:39.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-24T17:17:39.536 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-24T17:17:39.536 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:39.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-24T17:17:39.759 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:17:39.760 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:39.760 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:39.760 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:39.760 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:39.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-24T17:17:39.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:39.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:39.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-24T17:17:39.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-24T17:17:40.045 INFO:tasks.workunit.client.0.trial066.stderr:osd.5 is already in. 2026-02-24T17:17:40.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-24T17:17:40.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-24T17:17:40.341 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 is already in. 2026-02-24T17:17:40.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-24T17:17:40.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:17:40.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:17:40.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:17:40.354 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:17:40.354 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:17:40.354 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:17:40.354 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:17:40.354 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:17:40.354 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:17:40.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:17:40.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:17:40.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:17:40.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:17:40.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:17:40.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:8 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:40.671 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:17:40.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=21474836523 2026-02-24T17:17:40.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 21474836523 2026-02-24T17:17:40.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523' 2026-02-24T17:17:40.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:40.751 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:17:40.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=38654705706 2026-02-24T17:17:40.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 38654705706 2026-02-24T17:17:40.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706' 2026-02-24T17:17:40.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:40.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:17:40.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574890 2026-02-24T17:17:40.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574890 2026-02-24T17:17:40.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890' 2026-02-24T17:17:40.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:40.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:17:40.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=335007449116 2026-02-24T17:17:40.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 335007449116 2026-02-24T17:17:40.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116' 2026-02-24T17:17:40.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:40.991 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:17:41.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=98784247848 2026-02-24T17:17:41.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 98784247848 2026-02-24T17:17:41.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116 4-98784247848' 2026-02-24T17:17:41.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:41.068 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:17:41.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285593 2026-02-24T17:17:41.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285593 2026-02-24T17:17:41.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116 4-98784247848 5-356482285593' 2026-02-24T17:17:41.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:41.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:17:41.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=408021893141 2026-02-24T17:17:41.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 408021893141 2026-02-24T17:17:41.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116 4-98784247848 5-356482285593 6-408021893141' 2026-02-24T17:17:41.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:41.223 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:17:41.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=163208757286 2026-02-24T17:17:41.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 163208757286 2026-02-24T17:17:41.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116 4-98784247848 5-356482285593 6-408021893141 7-163208757286' 2026-02-24T17:17:41.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:41.299 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:17:41.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593765 2026-02-24T17:17:41.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593765 2026-02-24T17:17:41.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116 4-98784247848 5-356482285593 6-408021893141 7-163208757286 8-184683593765' 2026-02-24T17:17:41.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:41.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.9 flush_pg_stats 2026-02-24T17:17:41.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=433791696913 2026-02-24T17:17:41.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 433791696913 2026-02-24T17:17:41.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116 4-98784247848 5-356482285593 6-408021893141 7-163208757286 8-184683593765 9-433791696913' 2026-02-24T17:17:41.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:17:41.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.10 flush_pg_stats 2026-02-24T17:17:41.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=227633266724 2026-02-24T17:17:41.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 227633266724 2026-02-24T17:17:41.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-21474836523 1-38654705706 2-55834574890 3-335007449116 4-98784247848 5-356482285593 6-408021893141 7-163208757286 8-184683593765 9-433791696913 10-227633266724' 2026-02-24T17:17:41.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:41.538 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-21474836523 2026-02-24T17:17:41.538 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:41.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:17:41.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-21474836523 2026-02-24T17:17:41.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:41.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=21474836523 2026-02-24T17:17:41.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 21474836523' 2026-02-24T17:17:41.542 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 21474836523 2026-02-24T17:17:41.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:17:41.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 21474836523 -lt 21474836523 2026-02-24T17:17:41.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:41.758 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-38654705706 2026-02-24T17:17:41.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:41.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:17:41.760 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-38654705706 2026-02-24T17:17:41.760 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:41.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=38654705706 2026-02-24T17:17:41.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 38654705706' 2026-02-24T17:17:41.762 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 38654705706 2026-02-24T17:17:41.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:17:41.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 38654705706 -lt 38654705706 2026-02-24T17:17:41.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:41.985 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-55834574890 2026-02-24T17:17:41.986 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:41.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:17:41.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-55834574890 2026-02-24T17:17:41.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:41.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574890 2026-02-24T17:17:41.989 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 55834574890 2026-02-24T17:17:41.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 55834574890' 2026-02-24T17:17:41.989 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:17:42.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574890 -lt 55834574890 2026-02-24T17:17:42.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:42.217 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-335007449116 2026-02-24T17:17:42.217 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:42.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:17:42.219 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-335007449116 2026-02-24T17:17:42.219 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:42.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=335007449116 2026-02-24T17:17:42.220 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 335007449116 2026-02-24T17:17:42.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 335007449116' 2026-02-24T17:17:42.221 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:17:42.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 335007449116 -lt 335007449116 2026-02-24T17:17:42.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:42.449 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-98784247848 2026-02-24T17:17:42.449 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:42.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:17:42.451 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-98784247848 2026-02-24T17:17:42.451 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:42.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=98784247848 2026-02-24T17:17:42.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 98784247848' 2026-02-24T17:17:42.453 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 98784247848 2026-02-24T17:17:42.453 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:17:42.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247846 -lt 98784247848 2026-02-24T17:17:42.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:17:43.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:17:43.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:17:43.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 98784247849 -lt 98784247848 2026-02-24T17:17:43.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:43.884 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-356482285593 2026-02-24T17:17:43.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:43.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:17:43.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-356482285593 2026-02-24T17:17:43.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:43.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285593 2026-02-24T17:17:43.889 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 356482285593 2026-02-24T17:17:43.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 356482285593' 2026-02-24T17:17:43.889 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:17:44.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285593 -lt 356482285593 2026-02-24T17:17:44.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:44.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-408021893141 2026-02-24T17:17:44.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:44.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:17:44.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-408021893141 2026-02-24T17:17:44.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:44.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=408021893141 2026-02-24T17:17:44.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 408021893141' 2026-02-24T17:17:44.115 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 408021893141 2026-02-24T17:17:44.116 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:17:44.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 408021893141 -lt 408021893141 2026-02-24T17:17:44.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:44.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-163208757286 2026-02-24T17:17:44.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:44.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:17:44.333 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-163208757286 2026-02-24T17:17:44.334 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:44.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=163208757286 2026-02-24T17:17:44.335 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.7 seq 163208757286 2026-02-24T17:17:44.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 163208757286' 2026-02-24T17:17:44.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:17:44.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 163208757287 -lt 163208757286 2026-02-24T17:17:44.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:44.546 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-184683593765 2026-02-24T17:17:44.546 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:44.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:17:44.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-184683593765 2026-02-24T17:17:44.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:44.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593765 2026-02-24T17:17:44.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 184683593765' 2026-02-24T17:17:44.550 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.8 seq 184683593765 2026-02-24T17:17:44.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:17:44.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593766 -lt 184683593765 2026-02-24T17:17:44.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:44.776 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 9-433791696913 2026-02-24T17:17:44.776 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:44.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=9 2026-02-24T17:17:44.778 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 9-433791696913 2026-02-24T17:17:44.778 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:44.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=433791696913 2026-02-24T17:17:44.779 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.9 seq 433791696913 2026-02-24T17:17:44.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.9 seq 433791696913' 2026-02-24T17:17:44.779 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-24T17:17:44.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 433791696913 -lt 433791696913 2026-02-24T17:17:44.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:17:44.998 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 10-227633266724 2026-02-24T17:17:44.998 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:17:44.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=10 2026-02-24T17:17:45.000 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 10-227633266724 2026-02-24T17:17:45.000 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:17:45.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=227633266724 2026-02-24T17:17:45.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.10 seq 227633266724' 2026-02-24T17:17:45.002 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.10 seq 227633266724 2026-02-24T17:17:45.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-24T17:17:45.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 227633266724 -lt 227633266724 2026-02-24T17:17:45.228 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:17:45.228 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:45.229 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:45.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-24T17:17:45.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:17:45.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:17:45.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:17:45.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:17:45.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:17:45.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:17:45.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:17:45.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-24T17:17:45.732 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:17:45.733 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:17:45.733 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:17:46.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-24T17:17:46.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:17:46.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:17:46.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-24T17:17:46.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:17:46.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:46.024 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-24T17:17:46.024 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr:10' 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-24T17:17:46.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:17:46.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-24T17:17:46.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-24T17:17:46.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:17:46.481 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:17:46.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-24T17:17:46.771 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-24T17:17:46.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-24T17:17:46.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-24T17:17:46.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-24T17:17:46.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-24T17:17:46.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-24T17:17:46.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-24T17:17:47.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-24T17:17:47.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-24T17:17:47.445 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-lrc-a' already exists 2026-02-24T17:17:47.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:48.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-24T17:17:48.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:48.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-24T17:17:48.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-24T17:17:48.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:48.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:48.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-24T17:17:48.506 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-24T17:17:48.506 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-24T17:17:48.506 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:48.507 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-24T17:17:48.507 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:48.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-24T17:17:48.724 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:48.724 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:17:48.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-24T17:17:48.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-24T17:17:48.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-24T17:17:48.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-24T17:17:48.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-24T17:17:48.969 INFO:tasks.workunit.client.0.trial066.stderr:osd.3 is already out. 2026-02-24T17:17:48.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-24T17:17:53.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-24T17:17:53.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-24T17:17:53.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:53.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-24T17:17:53.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-24T17:17:53.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:54.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-24T17:17:54.195 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:54.195 INFO:tasks.workunit.client.0.trial066.stderr:7' 2026-02-24T17:17:54.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-24T17:17:54.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:54.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:54.215 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-24T17:17:54.444 INFO:tasks.workunit.client.0.trial066.stderr:osd.3 is already in. 2026-02-24T17:17:54.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:17:54.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-24T17:17:54.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-24T17:17:54.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-24T17:17:54.742 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-lrc-a' does not exist 2026-02-24T17:17:54.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-24T17:17:55.023 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-24T17:17:55.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-24T17:17:55.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-24T17:17:55.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-24T17:17:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-24T17:17:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-24T17:17:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-24T17:17:55.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-24T17:17:55.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-24T17:17:55.682 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-lrc' already exists 2026-02-24T17:17:55.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:17:56.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-24T17:17:56.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:17:56.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-24T17:17:56.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-24T17:17:56.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:17:56.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:17:56.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-24T17:17:56.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-24T17:17:56.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-24T17:17:56.746 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:17:56.746 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-24T17:17:56.746 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:17:56.956 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:10 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:7' 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-24T17:17:56.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-24T17:17:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-24T17:17:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-24T17:17:56.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-24T17:17:57.207 INFO:tasks.workunit.client.0.trial066.stderr:osd.7 is already out. 2026-02-24T17:17:57.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-24T17:18:02.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-24T17:18:02.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-24T17:18:02.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:18:02.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-24T17:18:02.221 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-24T17:18:02.221 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:18:02.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-24T17:18:02.436 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:18:02.436 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:18:02.436 INFO:tasks.workunit.client.0.trial066.stderr:10 2026-02-24T17:18:02.437 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:18:02.437 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:18:02.437 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:18:02.437 INFO:tasks.workunit.client.0.trial066.stderr:4' 2026-02-24T17:18:02.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-24T17:18:02.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:18:02.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:18:02.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-24T17:18:02.687 INFO:tasks.workunit.client.0.trial066.stderr:osd.7 is already in. 2026-02-24T17:18:02.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:18:02.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-24T17:18:02.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-24T17:18:02.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-24T17:18:02.985 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-lrc' does not exist 2026-02-24T17:18:02.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-24T17:18:03.289 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile profile-lrc does not exist 2026-02-24T17:18:03.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-24T17:18:03.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-24T17:18:03.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-24T17:18:03.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-24T17:18:03.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-24T17:18:03.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-24T17:18:03.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-24T17:18:03.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-24T17:18:03.965 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-shec' already exists 2026-02-24T17:18:03.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-24T17:18:04.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-24T17:18:04.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:18:05.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:18:05.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-24T17:18:05.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-24T17:18:05.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-24T17:18:05.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:18:05.028 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-24T17:18:05.028 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:18:05.243 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-24T17:18:05.243 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:18:05.244 INFO:tasks.workunit.client.0.trial066.stderr:1' 2026-02-24T17:18:05.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-24T17:18:05.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-24T17:18:05.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-24T17:18:05.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-24T17:18:05.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-24T17:18:05.507 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 is already out. 2026-02-24T17:18:05.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-24T17:18:10.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-24T17:18:10.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-24T17:18:10.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-24T17:18:10.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-24T17:18:10.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-24T17:18:10.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:18:10.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-24T17:18:10.732 INFO:tasks.workunit.client.0.trial066.stderr:9 2026-02-24T17:18:10.732 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:18:10.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-24T17:18:10.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-24T17:18:10.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-24T17:18:10.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-24T17:18:10.978 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 is already in. 2026-02-24T17:18:10.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-24T17:18:10.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-24T17:18:10.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-24T17:18:10.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-24T17:18:11.282 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-shec' does not exist 2026-02-24T17:18:11.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-24T17:18:11.561 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile profile-shec does not exist 2026-02-24T17:18:11.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-24T17:18:11.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-24T17:18:11.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-24T17:18:11.834 INFO:tasks.workunit.client.0.trial066.stderr:pool 'ecpool' does not exist 2026-02-24T17:18:11.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-24T17:18:11.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-24T17:18:11.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:18:11.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-24T17:18:11.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:18:11.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:18:11.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:18:11.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:18:11.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:18:11.956 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:18:11.956 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:18:11.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:18:11.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:18:11.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:18:11.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:18:11.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:18:11.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:18:11.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:18:11.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:18:11.961 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:18:11.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:18:11.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:18:11.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-24T17:18:12.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:18:12.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:18:12.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.45680 2026-02-24T17:18:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:18:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:18:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/test-erasure-code 0 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:18:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:18:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:18:12.030 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:18:12.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:18:12.031 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:18:12.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:18:12.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:18:12.032 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:18:12.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:18:12.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:18:12.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:18:12.034 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:18:12.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:18:12.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-24T17:18:12.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-24T17:18:12.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:18:12.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.45680 2026-02-24T17:18:12.037 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.45680 2026-02-24T17:18:12.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:18:12.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:18:12.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-24T17:18:12.039 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T17:18:12.040 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T17:18:12.051 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-24T17:18:12.052 DEBUG:teuthology.orchestra.run.trial066:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=618c318a7a0b287ab5aeff207b7bf1d73ef45044 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-24T17:18:12.102 INFO:tasks.workunit.client.0.trial066.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-24T17:18:12.108 INFO:tasks.workunit.client.0.trial066.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: export PYTHONWARNINGS=ignore 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: main: PYTHONWARNINGS=ignore 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: export CEPH_CONF=/dev/null 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2424: main: CEPH_CONF=/dev/null 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: main: unset CEPH_ARGS 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2427: main: local code 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2428: main: run td/test-erasure-eio 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-24T17:18:12.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-24T17:18:12.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-24T17:18:12.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-24T17:18:12.110 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-24T17:18:12.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none ' 2026-02-24T17:18:12.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-24T17:18:12.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-24T17:18:12.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-24T17:18:12.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:TEST_ec_object_attr_read_error 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:TEST_ec_recovery_multiple_errors 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:TEST_ec_recovery_multiple_objects 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:TEST_ec_recovery_unfound 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:TEST_ec_single_recovery_error 2026-02-24T17:18:12.113 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_get_bad_size_shard_0 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_get_bad_size_shard_1 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_get_subread_missing 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:18:12.114 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:18:12.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:18:12.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:18:12.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:18:12.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:18:12.115 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:18:12.116 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:18:12.116 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:18:12.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:18:12.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:18:12.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:18:12.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:18:12.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:18:12.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:18:12.120 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:18:12.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:18:12.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:18:12.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:18:12.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:18:12.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:12.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:18:12.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:18:12.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:18:12.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:18:12.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:18:12.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:12.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:18:12.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:18:12.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-24T17:18:12.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-24T17:18:12.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:18:12.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:18:12.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:18:12.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:18:12.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:18:12.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:18:12.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:18:12.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:18:12.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:18:12.153 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:18:12.153 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:12.153 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:12.154 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:12.154 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.154 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:12.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:12.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:18:12.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:18:12.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:18:12.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:18:12.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:18:12.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:18:12.182 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:18:12.182 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:18:12.182 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:18:12.182 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:18:12.182 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:12.182 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.182 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:12.183 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:18:12.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:18:12.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:18:12.240 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:18:12.241 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:12.242 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.242 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:12.242 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:18:12.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:18:12.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:18:12.296 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:18:12.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:18:12.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:18:12.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:18:12.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:18:12.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:18:12.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:18:12.413 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:18:12.414 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:12.414 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:12.414 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:12.414 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:12.414 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:12.414 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:12.415 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:18:12.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:18:12.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:18:12.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:18:12.541 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:18:12.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:18:13.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:18:13.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:18:13.601 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:18:13.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:18:13.609 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:18:12.177+0000 7f883ccfbec0 0 load: jerasure load: lrc 2026-02-24T17:18:13.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-24T17:18:13.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:18:13.611 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-24T17:18:13.614 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:18:13.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:13.616 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:18:13.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:18:13.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:18:13.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:18:13.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:18:13.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:18:13.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8000d98d-ca86-4a92-810f-d0afd039c01a 2026-02-24T17:18:13.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8000d98d-ca86-4a92-810f-d0afd039c01a' 2026-02-24T17:18:13.619 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 8000d98d-ca86-4a92-810f-d0afd039c01a 2026-02-24T17:18:13.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:18:13.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBV3Z1pMQ8RJhAAKxh7kNbv/Ww0MiUZ07m5sA== 2026-02-24T17:18:13.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBV3Z1pMQ8RJhAAKxh7kNbv/Ww0MiUZ07m5sA=="}' 2026-02-24T17:18:13.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8000d98d-ca86-4a92-810f-d0afd039c01a -i td/test-erasure-eio/0/new.json 2026-02-24T17:18:13.746 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:18:13.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:18:13.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBV3Z1pMQ8RJhAAKxh7kNbv/Ww0MiUZ07m5sA== --osd-uuid 8000d98d-ca86-4a92-810f-d0afd039c01a 2026-02-24T17:18:13.774 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:13.781+0000 7f4d0cf33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:13.776 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:13.781+0000 7f4d0cf33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:13.777 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:13.781+0000 7f4d0cf33a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:13.777 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:13.781+0000 7f4d0cf33a40 -1 bdev(0x557a9ced8400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:18:13.777 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:13.781+0000 7f4d0cf33a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:18:14.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:18:14.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:18:14.508 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:18:14.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:18:14.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:18:14.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:18:14.653 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:18:14.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:18:14.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:18:14.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:18:14.654 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:18:14.671 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:14.677+0000 7f41336d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:14.673 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:14.677+0000 7f41336d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:14.673 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:14.681+0000 7f41336d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:14.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:18:14.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:18:14.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:18:14.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:18:14.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:18:14.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:14.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:18:14.765 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:18:14.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:14.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:18:14.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:15.091 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:15.097+0000 7f41336d9a40 -1 Falling back to public interface 2026-02-24T17:18:15.732 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:15.737+0000 7f41336d9a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:18:15.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:15.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:15.978 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:18:15.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:18:15.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:15.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:18:16.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:17.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:17.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:17.197 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:18:17.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:18:17.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:17.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:18:17.406 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1596224930,v1:127.0.0.1:6803/1596224930] [v2:127.0.0.1:6804/1596224930,v1:127.0.0.1:6805/1596224930] exists,up 8000d98d-ca86-4a92-810f-d0afd039c01a 2026-02-24T17:18:17.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:18:17.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:18:17.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:18:17.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:18:17.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:18:17.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:18:17.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:18:17.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:18:17.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:18:17.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:18:17.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:18:17.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:18:17.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:18:17.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:18:17.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:18:17.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:18:17.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:18:17.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:18:17.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:18:17.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:18:17.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:17.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:17.416 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:17.416 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:17.416 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:17.417 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:17.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:17.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:18:17.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:18:17.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:18:17.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:18:17.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:18:17.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:18:17.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:18:17.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:18:17.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:18:17.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:18:17.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:18:17.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:18:17.419 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:18:17.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3c300353-8248-4324-a406-5ced791457fb 2026-02-24T17:18:17.420 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 3c300353-8248-4324-a406-5ced791457fb 2026-02-24T17:18:17.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 3c300353-8248-4324-a406-5ced791457fb' 2026-02-24T17:18:17.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:18:17.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZ3Z1pujrvGRAA3jt9aF6ywwuTpXWx7NmFYQ== 2026-02-24T17:18:17.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZ3Z1pujrvGRAA3jt9aF6ywwuTpXWx7NmFYQ=="}' 2026-02-24T17:18:17.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3c300353-8248-4324-a406-5ced791457fb -i td/test-erasure-eio/1/new.json 2026-02-24T17:18:17.647 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:18:17.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:18:17.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZ3Z1pujrvGRAA3jt9aF6ywwuTpXWx7NmFYQ== --osd-uuid 3c300353-8248-4324-a406-5ced791457fb 2026-02-24T17:18:17.685 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:17.689+0000 7fa2027a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:17.687 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:17.693+0000 7fa2027a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:17.688 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:17.693+0000 7fa2027a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:17.689 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:17.693+0000 7fa2027a3a40 -1 bdev(0x5638ceb60400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:18:17.689 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:17.693+0000 7fa2027a3a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:18:18.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:18:18.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:18:18.354 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:18:18.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:18:18.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:18:18.647 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:18:18.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:18:18.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:18:18.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:18:18.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:18:18.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:18:18.673 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:18.677+0000 7f4645120a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:18.674 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:18.681+0000 7f4645120a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:18.676 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:18.681+0000 7f4645120a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:18.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:18:18.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:18:18.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:18:18.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:18:18.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:18:18.866 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:18:18.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:18.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:18:18.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:18.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:18:19.031 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:19.037+0000 7f4645120a40 -1 Falling back to public interface 2026-02-24T17:18:19.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:19.726 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:19.733+0000 7f4645120a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:18:20.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:20.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:20.074 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:18:20.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:18:20.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:20.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:18:20.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:21.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:21.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:21.295 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:18:21.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:18:21.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:21.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:18:21.511 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/286402404,v1:127.0.0.1:6811/286402404] [v2:127.0.0.1:6812/286402404,v1:127.0.0.1:6813/286402404] exists,up 3c300353-8248-4324-a406-5ced791457fb 2026-02-24T17:18:21.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:18:21.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:18:21.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:18:21.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:18:21.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:18:21.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:21.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:18:21.518 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:18:21.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5292e0a7-9d45-4791-a751-b2b021fcc8bd 2026-02-24T17:18:21.519 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 5292e0a7-9d45-4791-a751-b2b021fcc8bd 2026-02-24T17:18:21.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5292e0a7-9d45-4791-a751-b2b021fcc8bd' 2026-02-24T17:18:21.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:18:21.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBd3Z1pxc04IBAA2lzZAMrOqW0P8VNOXaRjEQ== 2026-02-24T17:18:21.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBd3Z1pxc04IBAA2lzZAMrOqW0P8VNOXaRjEQ=="}' 2026-02-24T17:18:21.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5292e0a7-9d45-4791-a751-b2b021fcc8bd -i td/test-erasure-eio/2/new.json 2026-02-24T17:18:21.742 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:18:21.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:18:21.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBd3Z1pxc04IBAA2lzZAMrOqW0P8VNOXaRjEQ== --osd-uuid 5292e0a7-9d45-4791-a751-b2b021fcc8bd 2026-02-24T17:18:21.771 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:21.777+0000 7f698e3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:21.773 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:21.777+0000 7f698e3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:21.774 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:21.781+0000 7f698e3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:21.775 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:21.781+0000 7f698e3d6a40 -1 bdev(0x55e1442d8400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:18:21.775 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:21.781+0000 7f698e3d6a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:18:22.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:18:22.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:18:22.441 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:18:22.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:18:22.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:18:22.731 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:18:22.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:18:22.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:18:22.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:18:22.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:18:22.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:18:22.757 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:22.761+0000 7f927f113a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:22.759 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:22.765+0000 7f927f113a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:22.760 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:22.765+0000 7f927f113a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:22.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:18:22.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:18:22.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:18:22.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:18:22.944 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:18:22.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:18:22.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:22.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:18:22.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:22.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:18:23.107 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:23.113+0000 7f927f113a40 -1 Falling back to public interface 2026-02-24T17:18:23.159 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:23.695 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:23.701+0000 7f927f113a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:18:24.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:24.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:24.161 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:18:24.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:18:24.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:24.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:18:24.371 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/984498933,v1:127.0.0.1:6819/984498933] [v2:127.0.0.1:6820/984498933,v1:127.0.0.1:6821/984498933] exists,up 5292e0a7-9d45-4791-a751-b2b021fcc8bd 2026-02-24T17:18:24.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:18:24.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:18:24.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:18:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:18:24.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:18:24.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:18:24.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=263b4337-1723-48eb-b024-5b1b6a6f2a21 2026-02-24T17:18:24.379 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 263b4337-1723-48eb-b024-5b1b6a6f2a21 2026-02-24T17:18:24.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 263b4337-1723-48eb-b024-5b1b6a6f2a21' 2026-02-24T17:18:24.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:18:24.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBg3Z1p7hTYFxAAdG5hIExGZQxP5AaI9hmH3Q== 2026-02-24T17:18:24.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBg3Z1p7hTYFxAAdG5hIExGZQxP5AaI9hmH3Q=="}' 2026-02-24T17:18:24.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 263b4337-1723-48eb-b024-5b1b6a6f2a21 -i td/test-erasure-eio/3/new.json 2026-02-24T17:18:24.617 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:18:24.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:18:24.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBg3Z1p7hTYFxAAdG5hIExGZQxP5AaI9hmH3Q== --osd-uuid 263b4337-1723-48eb-b024-5b1b6a6f2a21 2026-02-24T17:18:24.645 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:24.653+0000 7f15174a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:24.647 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:24.653+0000 7f15174a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:24.648 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:24.653+0000 7f15174a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:24.648 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:24.653+0000 7f15174a7a40 -1 bdev(0x55c4bc486400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:18:24.648 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:24.653+0000 7f15174a7a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:18:25.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:18:25.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:18:25.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:18:25.341 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:18:25.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:18:25.639 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:18:25.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:18:25.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:18:25.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:18:25.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:18:25.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:18:25.665 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:25.669+0000 7fd983b71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:25.667 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:25.673+0000 7fd983b71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:25.667 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:25.673+0000 7fd983b71a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:25.851 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:18:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:18:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:18:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:18:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:18:25.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:25.852 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:18:25.853 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:18:25.853 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:25.853 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:18:26.031 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:26.037+0000 7fd983b71a40 -1 Falling back to public interface 2026-02-24T17:18:26.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:26.638 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:26.645+0000 7fd983b71a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:18:27.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:27.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:27.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:18:27.071 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:18:27.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:27.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:18:27.298 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2655573368,v1:127.0.0.1:6827/2655573368] [v2:127.0.0.1:6828/2655573368,v1:127.0.0.1:6829/2655573368] exists,up 263b4337-1723-48eb-b024-5b1b6a6f2a21 2026-02-24T17:18:27.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:18:27.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-24T17:18:27.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:27.303 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:18:27.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:18:27.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:18:27.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:18:27.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:18:27.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:18:27.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-24T17:18:27.305 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:18:27.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4cc8907a-95fd-4f63-9e58-f2255cf147bd 2026-02-24T17:18:27.306 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 4cc8907a-95fd-4f63-9e58-f2255cf147bd 2026-02-24T17:18:27.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 4cc8907a-95fd-4f63-9e58-f2255cf147bd' 2026-02-24T17:18:27.307 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:18:27.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBj3Z1pWdOCExAAYwkexbz+6D6SlOzHMpYupA== 2026-02-24T17:18:27.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBj3Z1pWdOCExAAYwkexbz+6D6SlOzHMpYupA=="}' 2026-02-24T17:18:27.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4cc8907a-95fd-4f63-9e58-f2255cf147bd -i td/test-erasure-eio/4/new.json 2026-02-24T17:18:27.544 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:18:27.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-24T17:18:27.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBj3Z1pWdOCExAAYwkexbz+6D6SlOzHMpYupA== --osd-uuid 4cc8907a-95fd-4f63-9e58-f2255cf147bd 2026-02-24T17:18:27.572 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:27.577+0000 7fa71cf62a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:27.574 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:27.581+0000 7fa71cf62a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:27.575 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:27.581+0000 7fa71cf62a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:27.576 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:27.581+0000 7fa71cf62a40 -1 bdev(0x561e9c84a400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:18:27.576 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:27.581+0000 7fa71cf62a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-24T17:18:28.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-24T17:18:28.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:18:28.273 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:18:28.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:18:28.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:18:28.569 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:18:28.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:18:28.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:18:28.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:18:28.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:18:28.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:18:28.597 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:28.601+0000 7faaa92cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:28.599 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:28.605+0000 7faaa92cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:28.600 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:28.605+0000 7faaa92cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:28.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:18:28.982 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:28.989+0000 7faaa92cda40 -1 Falling back to public interface 2026-02-24T17:18:28.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:29.640 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:29.645+0000 7faaa92cda40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:18:30.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:30.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:30.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:18:30.002 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:18:30.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:30.003 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:18:30.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:31.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:31.219 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:18:31.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:31.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:18:31.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:31.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:18:31.430 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/807552493,v1:127.0.0.1:6835/807552493] [v2:127.0.0.1:6836/807552493,v1:127.0.0.1:6837/807552493] exists,up 4cc8907a-95fd-4f63-9e58-f2255cf147bd 2026-02-24T17:18:31.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:18:31.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:18:31.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:18:31.432 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:18:31.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:18:31.496 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:18:31.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:18:31.504 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:18:15.101+0000 7f41336d9a40 0 load: jerasure load: lrc 2026-02-24T17:18:31.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:18:31.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-24T17:18:31.785 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:18:31.785 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:18:32.158 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:18:32.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:18:33.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:18:33.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:18:33.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:18:33.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:18:33.171 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:18:33.172 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:18:33.172 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:18:33.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:18:33.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:18:33.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:18:33.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:18:33.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:18:33.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:18:33.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:18:33.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:18:33.271 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:18:33.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:18:33.484 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:18:33.484 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:18:33.484 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:18:33.484 INFO:tasks.workunit.client.0.trial066.stderr:4' 2026-02-24T17:18:33.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:18:33.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:18:33.484 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:18:33.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803782 2026-02-24T17:18:33.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803782 2026-02-24T17:18:33.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782' 2026-02-24T17:18:33.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:18:33.566 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:18:33.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607557 2026-02-24T17:18:33.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607557 2026-02-24T17:18:33.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557' 2026-02-24T17:18:33.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:18:33.647 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:18:33.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411332 2026-02-24T17:18:33.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411332 2026-02-24T17:18:33.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-77309411332' 2026-02-24T17:18:33.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:18:33.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:18:33.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149700 2026-02-24T17:18:33.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149700 2026-02-24T17:18:33.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-77309411332 3-111669149700' 2026-02-24T17:18:33.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:18:33.809 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:18:33.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=150323855363 2026-02-24T17:18:33.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 150323855363 2026-02-24T17:18:33.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-77309411332 3-111669149700 4-150323855363' 2026-02-24T17:18:33.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:18:33.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803782 2026-02-24T17:18:33.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:18:33.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:18:33.890 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803782 2026-02-24T17:18:33.890 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:18:33.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803782 2026-02-24T17:18:33.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-24T17:18:33.892 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803782 2026-02-24T17:18:33.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:18:34.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-24T17:18:34.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:18:35.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:18:35.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:18:35.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-24T17:18:35.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:18:35.326 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607557 2026-02-24T17:18:35.326 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:18:35.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:18:35.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607557 2026-02-24T17:18:35.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:18:35.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607557 2026-02-24T17:18:35.330 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607557 2026-02-24T17:18:35.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-24T17:18:35.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:18:35.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-24T17:18:35.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:18:35.546 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411332 2026-02-24T17:18:35.546 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:18:35.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:18:35.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411332 2026-02-24T17:18:35.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:18:35.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411332 2026-02-24T17:18:35.549 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411332 2026-02-24T17:18:35.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-24T17:18:35.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:18:35.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-24T17:18:35.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:18:35.762 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149700 2026-02-24T17:18:35.762 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:18:35.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:18:35.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149700 2026-02-24T17:18:35.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:18:35.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149700 2026-02-24T17:18:35.766 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 111669149700 2026-02-24T17:18:35.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-24T17:18:35.766 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:18:35.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-24T17:18:35.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:18:35.981 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-150323855363 2026-02-24T17:18:35.981 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:18:35.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:18:35.983 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-150323855363 2026-02-24T17:18:35.983 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:18:35.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=150323855363 2026-02-24T17:18:35.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 150323855363' 2026-02-24T17:18:35.985 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 150323855363 2026-02-24T17:18:35.985 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:18:36.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-24T17:18:36.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:18:36.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:18:36.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:18:36.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:18:36.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:18:36.481 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:18:36.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:18:36.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:18:36.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:18:36.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:18:36.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:18:36.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:18:36.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:18:36.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:18:36.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:18:36.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:18:36.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:18:36.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:18:36.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-24T17:18:37.175 INFO:tasks.workunit.client.0.trial066.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T17:18:37.175 INFO:tasks.workunit.client.0.trial066.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:33.653504+0000 0'0 42:13 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-24T17:18:32.115657+0000 0'0 2026-02-24T17:18:32.115657+0000 0 0 periodic scrub scheduled @ 2026-02-26T01:03:34.956740+0000 0 0 2026-02-24T17:18:37.175 INFO:tasks.workunit.client.0.trial066.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.628572+0000 0'0 43:23 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T18:50:41.864120+0000 0 0 2026-02-24T17:18:37.175 INFO:tasks.workunit.client.0.trial066.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.628667+0000 0'0 41:94 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:43:43.943720+0000 0 0 2026-02-24T17:18:37.176 INFO:tasks.workunit.client.0.trial066.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.719201+0000 0'0 43:23 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T19:03:10.066238+0000 0 0 2026-02-24T17:18:37.176 INFO:tasks.workunit.client.0.trial066.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:24.673012+0000 0'0 43:67 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:24:10.971677+0000 0 0 2026-02-24T17:18:37.176 INFO:tasks.workunit.client.0.trial066.stdout: 2026-02-24T17:18:37.176 INFO:tasks.workunit.client.0.trial066.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T17:18:37.177 INFO:tasks.workunit.client.0.trial066.stderr:dumped pgs 2026-02-24T17:18:37.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:18:37.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:18:37.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:18:37.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-24T17:18:37.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:37.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:18:37.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:37.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:18:37.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:37.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:18:37.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:18:37.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:18:37.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:37.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-24T17:18:37.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-24T17:18:37.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-24T17:18:37.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:18:37.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-24T17:18:37.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-24T17:18:37.434 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-24T17:18:37.434 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:18:37.434 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:18:37.434 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:18:37.435 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:18:37.657 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:18:37.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:18:37.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:18:37.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:18:37.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:18:37.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-24T17:18:38.181 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already down. 2026-02-24T17:18:38.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-24T17:18:38.435 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already out. 2026-02-24T17:18:38.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-24T17:18:38.661 INFO:tasks.workunit.client.0.trial066.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T17:18:38.661 INFO:tasks.workunit.client.0.trial066.stdout:2.0 1 0 0 0 0 4096 0 0 1 0 1 peering 2026-02-24T17:18:37.736617+0000 43'1 43:18 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-24T17:18:32.115657+0000 0'0 2026-02-24T17:18:32.115657+0000 0 0 no scrub is scheduled 0 0 2026-02-24T17:18:38.661 INFO:tasks.workunit.client.0.trial066.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.628572+0000 0'0 43:23 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T18:50:41.864120+0000 0 0 2026-02-24T17:18:38.661 INFO:tasks.workunit.client.0.trial066.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.628667+0000 0'0 43:98 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:43:43.943720+0000 0 0 2026-02-24T17:18:38.661 INFO:tasks.workunit.client.0.trial066.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.719201+0000 0'0 43:23 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T19:03:10.066238+0000 0 0 2026-02-24T17:18:38.661 INFO:tasks.workunit.client.0.trial066.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:24.673012+0000 0'0 43:67 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:24:10.971677+0000 0 0 2026-02-24T17:18:38.662 INFO:tasks.workunit.client.0.trial066.stdout: 2026-02-24T17:18:38.662 INFO:tasks.workunit.client.0.trial066.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T17:18:38.662 INFO:tasks.workunit.client.0.trial066.stderr:dumped pgs 2026-02-24T17:18:38.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-24T17:18:38.672 INFO:tasks.workunit.client.0.trial066.stderr:4+0 records in 2026-02-24T17:18:38.672 INFO:tasks.workunit.client.0.trial066.stderr:4+0 records out 2026-02-24T17:18:38.672 INFO:tasks.workunit.client.0.trial066.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 7.2835e-05 s, 56.2 MB/s 2026-02-24T17:18:38.672 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-24T17:18:38.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.791 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.791 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:38.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:38.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.950 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.950 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:39.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:39.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.104 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.194 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.194 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.767 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.767 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.847 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.847 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:40.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:40.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.209 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.209 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.322 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.322 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.422 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.422 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.914 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.914 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:41.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:41.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:42.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:42.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.704 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.704 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:43.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:43.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.058 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.581 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.581 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.632 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.632 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:44.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:44.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.249 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.582 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.582 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.855 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.855 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:18:45.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-24T17:18:45.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-24T17:18:45.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:18:45.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:18:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:18:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:18:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:18:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:18:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj250 2026-02-24T17:18:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:18:45.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:18:45.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:18:45.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T17:18:45.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:18:45.906 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj250 2026-02-24T17:18:45.906 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:18:45.907 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-24T17:18:45.907 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-24T17:18:45.907 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:2147483647' 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:18:46.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-24T17:18:46.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:18:46.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:18:46.120 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:18:46.121 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:46.121 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:46.121 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:46.121 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:18:46.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:18:46.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:18:46.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:18:46.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:18:46.182 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:18:46.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj250 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj250 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-24T17:18:46.242 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:18:46.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:18:46.461 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:18:46.461 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:18:46.461 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:18:46.461 INFO:tasks.workunit.client.0.trial066.stderr:2147483647' 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:18:46.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:18:46.463 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:18:46.463 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-24T17:18:46.463 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:18:46.463 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:18:46.464 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:46.464 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:46.464 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:46.464 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:18:46.464 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:18:46.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:18:46.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:18:46.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:18:46.525 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-24T17:18:46.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:18:46.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:18:46.526 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:18:46.526 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:46.526 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:46.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:18:46.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:18:46.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-24T17:18:46.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-24T17:18:46.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:18:46.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:18:46.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:46.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:18:46.587 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:18:46.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:18:46.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:18:46.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:18:46.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:18:46.606 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:46.613+0000 7f859a444a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:46.608 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:46.613+0000 7f859a444a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:46.608 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:46.613+0000 7f859a444a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:18:46.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:46.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:18:47.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:18:47.047 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:47.053+0000 7f859a444a40 -1 Falling back to public interface 2026-02-24T17:18:47.654 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:18:47.661+0000 7f859a444a40 -1 osd.2 43 log_to_monitors true 2026-02-24T17:18:48.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:18:48.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:18:48.012 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:18:48.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:18:48.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:18:48.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:18:48.231 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up out weight 0 up_from 49 up_thru 20 down_at 44 last_clean_interval [18,43) [v2:127.0.0.1:6818/3180208811,v1:127.0.0.1:6819/3180208811] [v2:127.0.0.1:6820/3180208811,v1:127.0.0.1:6821/3180208811] exists,up 5292e0a7-9d45-4791-a751-b2b021fcc8bd 2026-02-24T17:18:48.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:18:48.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:18:48.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:18:48.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-24T17:18:48.479 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already in. 2026-02-24T17:18:48.489 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-24T17:19:03.491 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-24T17:19:03.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-24T17:19:03.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-24T17:19:03.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-24T17:19:03.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-24T17:19:03.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-24T17:19:03.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-24T17:19:03.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-24T17:19:03.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-24T17:19:03.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-24T17:19:03.706 INFO:tasks.workunit.client.0.trial066.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-24T17:19:03.707 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-24T17:19:03.707 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-24T17:19:03.707 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-24T17:18:49.668489+0000 47'301 53:896 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-24T17:18:32.115657+0000 0'0 2026-02-24T17:18:32.115657+0000 0 0 no scrub is scheduled 0 0 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:48.539781+0000 0'0 53:61 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T22:31:14.112269+0000 0 0 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.628667+0000 0'0 53:116 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:43:43.943720+0000 0 0 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.719201+0000 0'0 53:43 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T19:03:10.066238+0000 0 0 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:48.637527+0000 0'0 53:104 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-26T01:09:37.600799+0000 0 0 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout: 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T17:19:03.921 INFO:tasks.workunit.client.0.trial066.stderr:dumped pgs 2026-02-24T17:19:03.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:19:03.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:19:03.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:19:03.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:19:03.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:19:03.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:19:04.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:19:04.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-24T17:19:09.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-24T17:19:09.439 INFO:tasks.workunit.client.0.trial066.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T17:19:09.440 INFO:tasks.workunit.client.0.trial066.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-24T17:19:04.104287+0000 47'301 55:908 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-24T17:18:32.115657+0000 0'0 2026-02-24T17:18:32.115657+0000 0 0 no scrub is scheduled 0 0 2026-02-24T17:19:09.440 INFO:tasks.workunit.client.0.trial066.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-24T17:19:04.101884+0000 0'0 55:69 [4,1] 4 [4,1] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-26T01:27:59.755171+0000 0 0 2026-02-24T17:19:09.440 INFO:tasks.workunit.client.0.trial066.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.628667+0000 0'0 55:120 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T21:43:43.943720+0000 0 0 2026-02-24T17:19:09.440 INFO:tasks.workunit.client.0.trial066.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:18:30.719201+0000 0'0 55:47 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T19:03:10.066238+0000 0 0 2026-02-24T17:19:09.440 INFO:tasks.workunit.client.0.trial066.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-24T17:19:04.101641+0000 0'0 55:112 [1,0] 1 [1,0] 1 0'0 2026-02-24T17:18:12.534511+0000 0'0 2026-02-24T17:18:12.534511+0000 0 0 periodic scrub scheduled @ 2026-02-25T18:28:19.266554+0000 0 0 2026-02-24T17:19:09.440 INFO:tasks.workunit.client.0.trial066.stdout: 2026-02-24T17:19:09.440 INFO:tasks.workunit.client.0.trial066.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T17:19:09.441 INFO:tasks.workunit.client.0.trial066.stderr:dumped pgs 2026-02-24T17:19:09.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout:{ 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout: "num_missing": 1, 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout: "num_unfound": 1, 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout: "objects": [ 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout: "oid": { 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout: "oid": "obj250", 2026-02-24T17:19:09.535 INFO:tasks.workunit.client.0.trial066.stdout: "key": "", 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "snapid": -2, 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "hash": 2249616407, 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "max": 0, 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "pool": 2, 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "namespace": "" 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "need": "47'251", 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "have": "0'0", 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "flags": "none", 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "locations": [ 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:19:09.536 INFO:tasks.workunit.client.0.trial066.stdout: ] 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: "state": "NotRecovering", 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: "available_might_have_unfound": true, 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: "might_have_unfound": [ 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "2(4)", 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: "status": "osd is down" 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout: "more": false 2026-02-24T17:19:09.537 INFO:tasks.workunit.client.0.trial066.stdout:} 2026-02-24T17:19:09.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-24T17:19:09.622 INFO:tasks.workunit.client.0.trial066.stdout:{ 2026-02-24T17:19:09.622 INFO:tasks.workunit.client.0.trial066.stdout: "snap_trimq": "[]", 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: "snap_trimq_len": 0, 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: "epoch": 55, 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.623 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: "acting_recovery_backfill": [ 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:19:09.624 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "info": { 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s0", 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "0", 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "47'301", 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "47'250", 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "47'200", 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 301, 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:19:09.625 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "55", 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 41, 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 41, 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 54, 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.626 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 41, 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 54, 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 54, 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 41, 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:19:09.627 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "version": "47'301", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 909, 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 55, 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:19:04.104287+0000", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:19:04.104287+0000", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:19:04.104287+0000", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:19:04.104287+0000", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:18:37.219256+0000", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:19:04.102340+0000", 2026-02-24T17:19:09.628 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:19:04.102340+0000", 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:19:04.104287+0000", 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:19:04.101062+0000", 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:19:04.100853+0000", 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 54, 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "47'200", 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "47'200", 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "created": 41, 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:19:09.629 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 200, 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:19:09.630 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:19:09.631 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 1232896, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 301, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 1505, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 1, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 1, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 303, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 1, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 301, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:19:09.632 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 301, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 1204, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:19:09.633 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:19:09.634 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.635 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:19:09.636 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 300 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),4(2)", 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 1 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:19:09.637 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.638 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "peer_info": [ 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "peer": "0(3)", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s3", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "3", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "47'301", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "47'301", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "47'200", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 301, 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:19:09.639 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 41, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 41, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 54, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 41, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 54, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 54, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 41, 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.640 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "version": "47'301", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 899, 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 54, 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+remapped", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.641 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:18:37.219256+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:18:37.836470+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:18:37.836375+0000", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 54, 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "47'200", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "47'200", 2026-02-24T17:19:09.642 INFO:tasks.workunit.client.0.trial066.stdout: "created": 41, 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 200, 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:19:09.643 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-25T23:51:30.476492+0000", 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:19:09.644 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 1232896, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 301, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 1505, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 0, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 301, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 0, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 301, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:19:09.645 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 301, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 1204, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:19:09.646 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:19:09.647 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.648 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:19:09.649 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 301 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:19:09.650 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "peer": "1(1)", 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s1", 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "1", 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "47'301", 2026-02-24T17:19:09.651 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "47'301", 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "47'200", 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 301, 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 41, 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 41, 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 54, 2026-02-24T17:19:09.652 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 41, 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 54, 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 54, 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 41, 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.653 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "version": "47'301", 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 331, 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 47, 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+remapped", 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.654 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:18:37.219256+0000", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:18:37.836470+0000", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:18:37.836375+0000", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 54, 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "47'200", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "47'200", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "created": 41, 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:19:09.655 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 200, 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:19:09.656 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-25T23:51:30.476492+0000", 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 1232896, 2026-02-24T17:19:09.657 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 301, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 1505, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 1, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 301, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 0, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 301, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:19:09.658 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 301, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 1204, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:19:09.659 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:19:09.660 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.661 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 301 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:19:09.662 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "peer": "4(2)", 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s2", 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "2", 2026-02-24T17:19:09.663 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "47'301", 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "47'301", 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "47'200", 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 301, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 41, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 41, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 54, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 41, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:19:09.664 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 54, 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 54, 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 41, 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "version": "47'301", 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 331, 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 47, 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+remapped", 2026-02-24T17:19:09.665 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:18:37.219256+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:18:37.837684+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:18:45.910228+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:18:37.836470+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:18:37.836375+0000", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 54, 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "47'200", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "47'200", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "created": 41, 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 42, 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:19:09.666 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:18:32.115657+0000", 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 200, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:19:09.667 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-25T23:51:30.476492+0000", 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 1232896, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 301, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 1505, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 301, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 301, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:19:09.668 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 301, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 1204, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:19:09.669 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.670 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: 2147483647 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 301 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.671 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 55, 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "recovery_state": [ 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.672 INFO:tasks.workunit.client.0.trial066.stdout: "name": "Started/Primary/Active", 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "enter_time": "2026-02-24T17:19:04.100877+0000", 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "might_have_unfound": [ 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "0(3)", 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "status": "already probed" 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "1(1)", 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "status": "already probed" 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "2(4)", 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "status": "osd is down" 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "4(2)", 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "status": "already probed" 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "recovery_progress": { 2026-02-24T17:19:09.673 INFO:tasks.workunit.client.0.trial066.stdout: "backfill_targets": [], 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "waiting_on_backfill": [], 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill_started": "MIN", 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "backfill_info": { 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "begin": "MIN", 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "end": "MIN", 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "objects": [] 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "peer_backfill_info": [], 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "backfills_in_flight": [], 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "recovering": [], 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "pg_backend": { 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "recovery_ops": [], 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "read_ops": [] 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "name": "Started", 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: "enter_time": "2026-02-24T17:19:04.000480+0000" 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.674 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "scrubber": { 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "active": false, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "must_scrub": false, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "must_deep_scrub": false, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "must_repair": false, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "sched_time": "2026-02-25T23:51:30.476492+0000", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "schedule": "not registered for scrubbing", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "urgency": "periodic-regular", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "shallow-target": { 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s0", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "level": "shallow", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "urgency": "periodic-regular", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "sched_time": "2026-02-25T23:51:30.476492+0000", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "orig_sched_time": "2026-02-25T23:51:30.476492+0000", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "last_issue": "ok", 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "forced": false, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "eligible": false, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "queued": false, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "active": false 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.675 INFO:tasks.workunit.client.0.trial066.stdout: "deep-target": { 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s0", 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "level": "deep", 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "urgency": "periodic-regular", 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "sched_time": "2026-03-01T22:54:08.781917+0000", 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "orig_sched_time": "2026-03-01T22:54:08.781917+0000", 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "last_issue": "ok", 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "forced": false, 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "eligible": false, 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "queued": false, 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "active": false 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout: "agent_state": {} 2026-02-24T17:19:09.676 INFO:tasks.workunit.client.0.trial066.stdout:} 2026-02-24T17:19:09.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:19:09.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-24T17:19:09.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:19:09.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-24T17:19:09.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-24T17:19:09.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-24T17:19:09.798 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:19:09.798 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-24T17:19:09.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-24T17:19:09.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-24T17:19:09.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-24T17:19:09.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:19:09.880 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-24T17:19:09.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:09.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:09.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:09.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:19:09.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:19:09.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:09.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:19:09.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:09.963 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:19:09.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:09.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:09.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:09.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:09.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:09.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:09.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:09.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:19:09.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:19:09.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:19:09.966 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:19:09.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:19:09.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:09.967 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:19:09.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:19:09.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:19:09.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:19:09.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:19:09.981 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:09.985+0000 7fbc8d31ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:09.982 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:09.989+0000 7fbc8d31ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:09.983 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:09.989+0000 7fbc8d31ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:10.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:19:10.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:10.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:19:10.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:10.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:10.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:10.178 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:10.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:10.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:10.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:19:10.319 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:10.325+0000 7fbc8d31ba40 -1 Falling back to public interface 2026-02-24T17:19:10.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:10.960 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:10.965+0000 7fbc8d31ba40 -1 osd.2 53 log_to_monitors true 2026-02-24T17:19:11.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:11.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:11.388 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:11.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:11.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:11.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:19:11.594 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 57 up_thru 20 down_at 54 last_clean_interval [49,53) [v2:127.0.0.1:6818/2406510247,v1:127.0.0.1:6819/2406510247] [v2:127.0.0.1:6820/2406510247,v1:127.0.0.1:6821/2406510247] exists,up 5292e0a7-9d45-4791-a751-b2b021fcc8bd 2026-02-24T17:19:11.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:11.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:11.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:11.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-24T17:19:16.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-24T17:19:16.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-24T17:19:16.671 INFO:tasks.workunit.client.0.trial066.stderr:pg has no unfound objects 2026-02-24T17:19:16.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-24T17:19:16.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:19:16.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:19:16.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:19:16.679 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:19:16.679 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:19:16.680 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:19:16.680 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:19:16.680 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:19:16.680 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:19:16.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:19:16.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:19:16.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:19:16.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:19:16.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:19:16.789 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:19:17.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:19:17.006 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:19:17.006 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:19:17.006 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:19:17.006 INFO:tasks.workunit.client.0.trial066.stderr:4' 2026-02-24T17:19:17.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:19:17.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:17.006 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:19:17.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803793 2026-02-24T17:19:17.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803793 2026-02-24T17:19:17.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793' 2026-02-24T17:19:17.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:17.084 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:19:17.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607568 2026-02-24T17:19:17.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607568 2026-02-24T17:19:17.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-51539607568' 2026-02-24T17:19:17.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:17.162 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:19:17.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=244813135876 2026-02-24T17:19:17.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 244813135876 2026-02-24T17:19:17.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-51539607568 2-244813135876' 2026-02-24T17:19:17.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:17.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:19:17.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149710 2026-02-24T17:19:17.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149710 2026-02-24T17:19:17.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-51539607568 2-244813135876 3-111669149710' 2026-02-24T17:19:17.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:17.319 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:19:17.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=150323855374 2026-02-24T17:19:17.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 150323855374 2026-02-24T17:19:17.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803793 1-51539607568 2-244813135876 3-111669149710 4-150323855374' 2026-02-24T17:19:17.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:17.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803793 2026-02-24T17:19:17.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:17.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:19:17.398 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803793 2026-02-24T17:19:17.398 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:17.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803793 2026-02-24T17:19:17.400 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803793 2026-02-24T17:19:17.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803793' 2026-02-24T17:19:17.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:19:17.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803790 -lt 25769803793 2026-02-24T17:19:17.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:19:18.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:19:18.616 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:19:18.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803793 -lt 25769803793 2026-02-24T17:19:18.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:18.827 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607568 2026-02-24T17:19:18.827 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:18.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:19:18.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607568 2026-02-24T17:19:18.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:18.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607568 2026-02-24T17:19:18.830 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607568 2026-02-24T17:19:18.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607568' 2026-02-24T17:19:18.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:19:19.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607568 -lt 51539607568 2026-02-24T17:19:19.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:19.051 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-244813135876 2026-02-24T17:19:19.052 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:19.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:19:19.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-244813135876 2026-02-24T17:19:19.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:19.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=244813135876 2026-02-24T17:19:19.055 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 244813135876 2026-02-24T17:19:19.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 244813135876' 2026-02-24T17:19:19.055 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:19:19.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 244813135876 -lt 244813135876 2026-02-24T17:19:19.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:19.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149710 2026-02-24T17:19:19.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:19.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:19:19.272 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149710 2026-02-24T17:19:19.272 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:19.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149710 2026-02-24T17:19:19.274 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 111669149710 2026-02-24T17:19:19.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149710' 2026-02-24T17:19:19.274 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:19:19.490 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149711 -lt 111669149710 2026-02-24T17:19:19.490 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:19.490 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-150323855374 2026-02-24T17:19:19.490 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:19.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:19:19.492 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-150323855374 2026-02-24T17:19:19.492 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:19.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=150323855374 2026-02-24T17:19:19.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 150323855374' 2026-02-24T17:19:19.494 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 150323855374 2026-02-24T17:19:19.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:19:19.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 150323855374 -lt 150323855374 2026-02-24T17:19:19.710 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:19:19.711 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:19:19.711 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:19:20.003 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:19:20.003 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:19:20.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:19:20.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:19:20.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:19:20.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:19:20.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:19:20.004 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:19:20.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:19:20.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:19:20.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:19:20.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:19:20.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:19:20.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:19:20.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:19:20.500 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-24T17:19:20.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-24T17:19:20.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-24T17:19:20.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-24T17:19:20.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-24T17:19:20.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-24T17:19:20.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-24T17:19:20.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-24T17:19:20.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-24T17:19:20.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-24T17:19:20.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-24T17:19:20.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.711 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-24T17:19:20.711 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-24T17:19:20.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-24T17:19:20.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.774 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.774 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-24T17:19:20.774 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.791 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-24T17:19:20.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-24T17:19:20.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-24T17:19:20.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-24T17:19:20.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-24T17:19:20.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-24T17:19:20.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-24T17:19:20.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-24T17:19:20.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-24T17:19:20.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:20.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:20.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-24T17:19:20.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-24T17:19:21.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-24T17:19:21.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-24T17:19:21.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-24T17:19:21.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-24T17:19:21.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-24T17:19:21.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-24T17:19:21.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.159 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-24T17:19:21.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-24T17:19:21.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-24T17:19:21.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-24T17:19:21.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-24T17:19:21.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-24T17:19:21.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-24T17:19:21.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-24T17:19:21.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-24T17:19:21.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-24T17:19:21.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.393 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-24T17:19:21.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.421 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.421 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-24T17:19:21.422 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-24T17:19:21.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-24T17:19:21.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-24T17:19:21.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-24T17:19:21.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-24T17:19:21.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-24T17:19:21.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-24T17:19:21.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-24T17:19:21.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-24T17:19:21.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-24T17:19:21.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-24T17:19:21.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-24T17:19:21.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-24T17:19:21.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-24T17:19:21.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-24T17:19:21.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-24T17:19:21.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.785 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-24T17:19:21.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-24T17:19:21.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-24T17:19:21.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.847 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-24T17:19:21.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-24T17:19:21.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-24T17:19:21.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.912 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.912 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-24T17:19:21.912 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-24T17:19:21.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-24T17:19:21.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-24T17:19:21.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:21.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:21.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-24T17:19:21.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-24T17:19:22.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-24T17:19:22.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-24T17:19:22.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-24T17:19:22.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-24T17:19:22.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-24T17:19:22.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-24T17:19:22.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-24T17:19:22.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-24T17:19:22.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-24T17:19:22.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-24T17:19:22.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-24T17:19:22.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-24T17:19:22.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-24T17:19:22.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-24T17:19:22.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-24T17:19:22.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-24T17:19:22.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-24T17:19:22.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-24T17:19:22.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-24T17:19:22.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-24T17:19:22.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-24T17:19:22.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.480 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-24T17:19:22.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-24T17:19:22.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-24T17:19:22.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-24T17:19:22.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-24T17:19:22.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-24T17:19:22.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-24T17:19:22.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-24T17:19:22.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-24T17:19:22.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-24T17:19:22.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-24T17:19:22.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-24T17:19:22.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-24T17:19:22.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-24T17:19:22.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-24T17:19:22.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-24T17:19:22.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-24T17:19:22.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-24T17:19:22.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-24T17:19:22.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-24T17:19:22.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-24T17:19:22.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-24T17:19:22.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:22.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:22.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-24T17:19:22.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-24T17:19:22.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-24T17:19:23.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-24T17:19:23.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-24T17:19:23.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-24T17:19:23.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-24T17:19:23.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-24T17:19:23.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-24T17:19:23.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.164 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-24T17:19:23.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-24T17:19:23.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-24T17:19:23.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-24T17:19:23.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-24T17:19:23.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-24T17:19:23.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-24T17:19:23.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-24T17:19:23.333 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-24T17:19:23.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-24T17:19:23.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-24T17:19:23.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-24T17:19:23.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-24T17:19:23.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.463 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.464 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.464 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-24T17:19:23.464 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-24T17:19:23.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-24T17:19:23.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.532 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-24T17:19:23.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-24T17:19:23.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.580 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.580 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-24T17:19:23.581 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-24T17:19:23.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-24T17:19:23.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-24T17:19:23.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-24T17:19:23.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-24T17:19:23.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-24T17:19:23.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-24T17:19:23.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-24T17:19:23.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-24T17:19:23.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.800 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-24T17:19:23.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-24T17:19:23.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.851 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-24T17:19:23.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-24T17:19:23.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-24T17:19:23.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-24T17:19:23.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-24T17:19:23.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.972 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.972 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-24T17:19:23.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-24T17:19:23.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:23.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:23.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-24T17:19:23.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-24T17:19:24.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-24T17:19:24.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-24T17:19:24.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-24T17:19:24.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-24T17:19:24.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-24T17:19:24.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-24T17:19:24.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-24T17:19:24.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-24T17:19:24.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-24T17:19:24.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-24T17:19:24.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-24T17:19:24.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-24T17:19:24.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-24T17:19:24.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-24T17:19:24.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-24T17:19:24.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-24T17:19:24.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.373 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-24T17:19:24.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-24T17:19:24.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-24T17:19:24.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-24T17:19:24.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-24T17:19:24.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-24T17:19:24.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-24T17:19:24.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-24T17:19:24.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-24T17:19:24.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-24T17:19:24.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-24T17:19:24.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-24T17:19:24.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-24T17:19:24.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-24T17:19:24.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-24T17:19:24.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-24T17:19:24.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-24T17:19:24.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-24T17:19:24.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-24T17:19:24.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.806 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.806 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-24T17:19:24.806 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-24T17:19:24.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-24T17:19:24.846 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-24T17:19:24.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-24T17:19:24.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-24T17:19:24.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-24T17:19:24.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-24T17:19:24.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-24T17:19:24.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-24T17:19:24.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:24.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:24.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-24T17:19:24.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-24T17:19:25.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-24T17:19:25.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-24T17:19:25.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-24T17:19:25.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-24T17:19:25.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-24T17:19:25.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-24T17:19:25.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-24T17:19:25.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-24T17:19:25.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-24T17:19:25.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-24T17:19:25.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-24T17:19:25.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.296 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.296 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-24T17:19:25.296 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-24T17:19:25.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-24T17:19:25.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-24T17:19:25.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-24T17:19:25.389 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-24T17:19:25.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-24T17:19:25.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-24T17:19:25.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.472 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-24T17:19:25.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-24T17:19:25.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-24T17:19:25.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-24T17:19:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-24T17:19:25.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-24T17:19:25.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-24T17:19:25.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-24T17:19:25.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-24T17:19:25.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-24T17:19:25.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-24T17:19:25.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-24T17:19:25.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-24T17:19:25.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-24T17:19:25.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-24T17:19:25.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-24T17:19:25.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-24T17:19:25.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-24T17:19:25.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-24T17:19:25.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-24T17:19:25.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.937 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-24T17:19:25.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-24T17:19:25.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-24T17:19:25.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-24T17:19:25.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:25.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:25.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-24T17:19:25.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-24T17:19:26.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.039 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.039 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-24T17:19:26.039 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-24T17:19:26.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.079 INFO:tasks.workunit.client.0.trial066.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-24T17:19:26.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-24T17:19:26.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-24T17:19:26.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-24T17:19:26.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-24T17:19:26.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-24T17:19:26.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-24T17:19:26.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-24T17:19:26.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-24T17:19:26.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-24T17:19:26.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-24T17:19:26.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-24T17:19:26.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-24T17:19:26.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-24T17:19:26.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-24T17:19:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-24T17:19:26.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-24T17:19:26.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-24T17:19:26.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-24T17:19:26.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-24T17:19:26.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-24T17:19:26.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-24T17:19:26.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.565 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-24T17:19:26.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-24T17:19:26.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-24T17:19:26.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-24T17:19:26.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-24T17:19:26.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-24T17:19:26.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-24T17:19:26.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-24T17:19:26.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-24T17:19:26.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-24T17:19:26.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-24T17:19:26.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.817 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-24T17:19:26.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-24T17:19:26.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-24T17:19:26.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-24T17:19:26.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-24T17:19:26.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-24T17:19:26.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-24T17:19:26.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-24T17:19:26.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:26.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:26.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-24T17:19:26.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-24T17:19:27.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-24T17:19:27.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-24T17:19:27.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-24T17:19:27.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.085 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.086 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.086 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-24T17:19:27.087 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-24T17:19:27.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-24T17:19:27.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-24T17:19:27.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-24T17:19:27.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:19:27.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-24T17:19:27.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-24T17:19:27.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:19:27.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:19:27.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:19:27.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:19:27.487 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:19:27.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:19:27.759 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:19:27.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:19:27.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:19:27.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:19:27.880 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:19:27.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:19:27.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:19:27.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:19:27.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:19:27.883 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:19:27.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:19:27.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:19:27.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:19:27.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:19:27.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:19:27.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:19:27.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:19:27.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:19:27.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:27.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:27.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:19:27.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:19:27.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:19:27.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:19:27.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:19:27.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:19:27.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:19:27.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:19:27.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:19:27.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:19:27.935 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:19:27.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:19:27.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:19:27.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:19:27.937 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:19:27.937 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:19:27.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:19:27.938 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:19:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:19:27.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:19:27.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:19:27.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:19:27.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:27.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:27.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:19:27.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:19:27.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:19:27.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:19:27.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:19:27.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:27.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:27.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:19:27.947 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:19:27.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:19:27.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:19:27.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:19:27.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:19:27.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:19:27.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:19:27.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:19:27.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:19:27.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:19:27.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:19:27.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:19:27.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:27.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:27.970 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:27.970 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:27.970 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:27.971 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:19:27.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:19:27.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:19:27.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:19:27.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:19:28.000 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:19:28.000 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:19:28.000 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:19:28.000 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:19:28.000 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:19:28.001 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:19:28.001 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:28.002 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:28.002 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:19:28.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:19:28.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:19:28.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:19:28.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:19:28.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:19:28.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:19:28.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:19:28.057 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:19:28.057 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:19:28.057 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:19:28.057 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:19:28.057 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:28.057 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:28.057 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:19:28.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:19:28.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:19:28.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:19:28.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:19:28.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:19:28.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:19:28.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:19:28.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:19:28.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:19:28.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:19:28.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:28.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:28.227 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:28.227 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:28.228 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:28.228 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:28.228 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:19:28.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:19:28.247 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:19:28.247 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:19:28.353 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:19:28.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:19:29.363 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:19:29.363 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:19:29.364 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:19:29.364 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:19:29.364 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:29.364 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:29.364 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:19:29.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:19:29.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:19:29.413 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:19:29.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:19:29.421 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:19:27.996+0000 7f2072a5eec0 0 load: jerasure load: lrc 2026-02-24T17:19:29.422 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-24T17:19:29.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-24T17:19:29.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-24T17:19:29.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-24T17:19:29.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-24T17:19:29.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:19:29.423 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-24T17:19:29.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:29.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:29.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:19:29.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:19:29.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:19:29.430 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:19:29.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=028acc4d-7d74-470e-8f95-e6df271ad334 2026-02-24T17:19:29.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 028acc4d-7d74-470e-8f95-e6df271ad334' 2026-02-24T17:19:29.431 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 028acc4d-7d74-470e-8f95-e6df271ad334 2026-02-24T17:19:29.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:19:29.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCh3Z1puW3BGhAAxTl+8guFWqPSDkuBV8E4Ig== 2026-02-24T17:19:29.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCh3Z1puW3BGhAAxTl+8guFWqPSDkuBV8E4Ig=="}' 2026-02-24T17:19:29.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 028acc4d-7d74-470e-8f95-e6df271ad334 -i td/test-erasure-eio/0/new.json 2026-02-24T17:19:29.550 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:29.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:19:29.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCh3Z1puW3BGhAAxTl+8guFWqPSDkuBV8E4Ig== --osd-uuid 028acc4d-7d74-470e-8f95-e6df271ad334 2026-02-24T17:19:29.576 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:29.584+0000 7fe21eca2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:29.577 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:29.584+0000 7fe21eca2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:29.578 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:29.588+0000 7fe21eca2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:29.579 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:29.588+0000 7fe21eca2a40 -1 bdev(0x5651fd82a400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:19:29.579 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:29.588+0000 7fe21eca2a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:19:30.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:19:30.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:19:30.345 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:19:30.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:19:30.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:19:30.484 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:19:30.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:19:30.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:30.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:19:30.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:19:30.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:19:30.500 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:30.508+0000 7f39447bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:30.502 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:30.508+0000 7f39447bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:30.503 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:30.512+0000 7f39447bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:30.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:30.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:19:30.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:30.943 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:30.952+0000 7f39447bea40 -1 Falling back to public interface 2026-02-24T17:19:31.575 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:31.584+0000 7f39447bea40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:19:31.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:31.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:31.813 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:31.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:31.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:31.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:19:32.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:33.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:33.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:33.022 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:19:33.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:19:33.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:33.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:19:33.238 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3305797854,v1:127.0.0.1:6803/3305797854] [v2:127.0.0.1:6804/3305797854,v1:127.0.0.1:6805/3305797854] exists,up 028acc4d-7d74-470e-8f95-e6df271ad334 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:19:33.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:33.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:33.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:19:33.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:19:33.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:19:33.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:19:33.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=dbb1ab40-8aef-46f9-840f-9d70f57df869 2026-02-24T17:19:33.244 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 dbb1ab40-8aef-46f9-840f-9d70f57df869 2026-02-24T17:19:33.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 dbb1ab40-8aef-46f9-840f-9d70f57df869' 2026-02-24T17:19:33.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:19:33.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCl3Z1pbubdDxAA44Me3gw8Oq214Exhjnn6tQ== 2026-02-24T17:19:33.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCl3Z1pbubdDxAA44Me3gw8Oq214Exhjnn6tQ=="}' 2026-02-24T17:19:33.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new dbb1ab40-8aef-46f9-840f-9d70f57df869 -i td/test-erasure-eio/1/new.json 2026-02-24T17:19:33.475 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:33.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:19:33.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCl3Z1pbubdDxAA44Me3gw8Oq214Exhjnn6tQ== --osd-uuid dbb1ab40-8aef-46f9-840f-9d70f57df869 2026-02-24T17:19:33.502 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:33.512+0000 7f6cf643ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:33.504 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:33.512+0000 7f6cf643ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:33.505 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:33.512+0000 7f6cf643ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:33.505 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:33.512+0000 7f6cf643ba40 -1 bdev(0x56405167c400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:19:33.505 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:33.512+0000 7f6cf643ba40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:19:34.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:19:34.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:19:34.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:19:34.156 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:19:34.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:19:34.459 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:19:34.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:19:34.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:34.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:19:34.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:19:34.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:19:34.475 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:34.484+0000 7f1f06dcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:34.477 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:34.484+0000 7f1f06dcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:34.478 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:34.488+0000 7f1f06dcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:34.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:19:34.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:34.678 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:19:34.851 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:34.860+0000 7f1f06dcea40 -1 Falling back to public interface 2026-02-24T17:19:34.895 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:35.539 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:35.548+0000 7f1f06dcea40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:19:35.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:35.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:35.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:35.898 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:35.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:35.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:19:36.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:37.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:37.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:37.110 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:19:37.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:19:37.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:37.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:19:37.329 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/132057751,v1:127.0.0.1:6811/132057751] [v2:127.0.0.1:6812/132057751,v1:127.0.0.1:6813/132057751] exists,up dbb1ab40-8aef-46f9-840f-9d70f57df869 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:19:37.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:37.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:19:37.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:19:37.333 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f6ed1084-a77d-45ac-b4ea-1330f8566b77 2026-02-24T17:19:37.333 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 f6ed1084-a77d-45ac-b4ea-1330f8566b77 2026-02-24T17:19:37.334 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 f6ed1084-a77d-45ac-b4ea-1330f8566b77' 2026-02-24T17:19:37.334 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:19:37.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCp3Z1pxbFHFRAAeJyIGx66zyliMYAOFZ6Rkw== 2026-02-24T17:19:37.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCp3Z1pxbFHFRAAeJyIGx66zyliMYAOFZ6Rkw=="}' 2026-02-24T17:19:37.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f6ed1084-a77d-45ac-b4ea-1330f8566b77 -i td/test-erasure-eio/2/new.json 2026-02-24T17:19:37.560 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:19:37.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:19:37.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCp3Z1pxbFHFRAAeJyIGx66zyliMYAOFZ6Rkw== --osd-uuid f6ed1084-a77d-45ac-b4ea-1330f8566b77 2026-02-24T17:19:37.587 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:37.596+0000 7fb5e93e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:37.589 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:37.596+0000 7fb5e93e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:37.590 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:37.600+0000 7fb5e93e5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:37.591 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:37.600+0000 7fb5e93e5a40 -1 bdev(0x55f516b24400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:19:37.591 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:37.600+0000 7fb5e93e5a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:19:38.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:19:38.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:19:38.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:19:38.269 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:19:38.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:19:38.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:19:38.555 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:19:38.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:38.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:19:38.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:19:38.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:19:38.579 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:38.588+0000 7f64879c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:38.581 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:38.588+0000 7f64879c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:38.582 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:38.592+0000 7f64879c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:38.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:19:38.939 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:38.948+0000 7f64879c0a40 -1 Falling back to public interface 2026-02-24T17:19:38.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:39.554 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:39.564+0000 7f64879c0a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:19:39.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:39.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:39.989 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:39.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:39.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:39.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:19:40.203 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4161604229,v1:127.0.0.1:6819/4161604229] [v2:127.0.0.1:6820/4161604229,v1:127.0.0.1:6821/4161604229] exists,up f6ed1084-a77d-45ac-b4ea-1330f8566b77 2026-02-24T17:19:40.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:40.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:40.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:40.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:19:40.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:19:40.209 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:19:40.209 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:19:40.209 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=719494eb-c7c7-426a-a58d-f03a223497ff 2026-02-24T17:19:40.209 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 719494eb-c7c7-426a-a58d-f03a223497ff' 2026-02-24T17:19:40.209 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 719494eb-c7c7-426a-a58d-f03a223497ff 2026-02-24T17:19:40.209 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:19:40.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCs3Z1p2q/PDRAAGGIGdjPT8k60ZQ44jvUIwQ== 2026-02-24T17:19:40.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCs3Z1p2q/PDRAAGGIGdjPT8k60ZQ44jvUIwQ=="}' 2026-02-24T17:19:40.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 719494eb-c7c7-426a-a58d-f03a223497ff -i td/test-erasure-eio/3/new.json 2026-02-24T17:19:40.444 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:19:40.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:19:40.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCs3Z1p2q/PDRAAGGIGdjPT8k60ZQ44jvUIwQ== --osd-uuid 719494eb-c7c7-426a-a58d-f03a223497ff 2026-02-24T17:19:40.480 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:40.488+0000 7fab5b853a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:40.481 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:40.492+0000 7fab5b853a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:40.483 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:40.492+0000 7fab5b853a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:40.483 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:40.492+0000 7fab5b853a40 -1 bdev(0x5592a5004400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:19:40.483 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:40.492+0000 7fab5b853a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:19:41.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:19:41.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:19:41.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:19:41.110 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:19:41.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:19:41.409 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:19:41.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:19:41.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:41.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:19:41.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:19:41.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:19:41.433 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:41.440+0000 7f236c90aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:41.435 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:41.444+0000 7f236c90aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:41.436 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:41.444+0000 7f236c90aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:41.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:19:41.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:41.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:19:41.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:41.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:41.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:41.616 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:41.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:41.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:41.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:19:41.811 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:41.820+0000 7f236c90aa40 -1 Falling back to public interface 2026-02-24T17:19:41.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:42.464 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:42.472+0000 7f236c90aa40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:19:42.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:42.835 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:42.835 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:42.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:42.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:42.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:19:43.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:44.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:44.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:44.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:19:44.053 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:19:44.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:44.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 26 up_thru 29 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4244770696,v1:127.0.0.1:6827/4244770696] [v2:127.0.0.1:6828/4244770696,v1:127.0.0.1:6829/4244770696] exists,up 719494eb-c7c7-426a-a58d-f03a223497ff 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:44.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:44.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:19:44.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-24T17:19:44.272 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:19:44.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ddd08971-9274-4b9d-81e0-b7d09f37c0e9 2026-02-24T17:19:44.273 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 ddd08971-9274-4b9d-81e0-b7d09f37c0e9 2026-02-24T17:19:44.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 ddd08971-9274-4b9d-81e0-b7d09f37c0e9' 2026-02-24T17:19:44.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:19:44.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCw3Z1p8AmfERAAYyPW1j6UZZQey6XdKFEwsA== 2026-02-24T17:19:44.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCw3Z1p8AmfERAAYyPW1j6UZZQey6XdKFEwsA=="}' 2026-02-24T17:19:44.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ddd08971-9274-4b9d-81e0-b7d09f37c0e9 -i td/test-erasure-eio/4/new.json 2026-02-24T17:19:44.511 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:19:44.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-24T17:19:44.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCw3Z1p8AmfERAAYyPW1j6UZZQey6XdKFEwsA== --osd-uuid ddd08971-9274-4b9d-81e0-b7d09f37c0e9 2026-02-24T17:19:44.539 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:44.548+0000 7f0a34561a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:44.540 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:44.548+0000 7f0a34561a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:44.541 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:44.548+0000 7f0a34561a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:44.541 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:44.548+0000 7f0a34561a40 -1 bdev(0x55bc870c2400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:19:44.541 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:44.552+0000 7f0a34561a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-24T17:19:45.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-24T17:19:45.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:19:45.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:19:45.150 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:19:45.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:19:45.449 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:19:45.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:19:45.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:45.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:19:45.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:19:45.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:19:45.473 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:45.480+0000 7f39c259ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:45.475 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:45.484+0000 7f39c259ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:45.476 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:45.484+0000 7f39c259ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:19:45.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:45.879 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:45.888+0000 7f39c259ca40 -1 Falling back to public interface 2026-02-24T17:19:46.435 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:46.444+0000 7f39c259ca40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:19:46.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:46.880 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:46.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:46.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:46.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:46.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:19:47.093 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/363162169,v1:127.0.0.1:6835/363162169] [v2:127.0.0.1:6836/363162169,v1:127.0.0.1:6837/363162169] exists,up ddd08971-9274-4b9d-81e0-b7d09f37c0e9 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:47.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:47.095 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:19:47.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-24T17:19:47.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:19:47.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9bfc0c25-4ce2-42c0-8877-0fe11dc1969f 2026-02-24T17:19:47.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 9bfc0c25-4ce2-42c0-8877-0fe11dc1969f' 2026-02-24T17:19:47.097 INFO:tasks.workunit.client.0.trial066.stdout:add osd5 9bfc0c25-4ce2-42c0-8877-0fe11dc1969f 2026-02-24T17:19:47.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:19:47.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCz3Z1p3p4nBxAAbW2FrXHu8UpPVHiUgcjYTA== 2026-02-24T17:19:47.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCz3Z1p3p4nBxAAbW2FrXHu8UpPVHiUgcjYTA=="}' 2026-02-24T17:19:47.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9bfc0c25-4ce2-42c0-8877-0fe11dc1969f -i td/test-erasure-eio/5/new.json 2026-02-24T17:19:47.324 INFO:tasks.workunit.client.0.trial066.stdout:5 2026-02-24T17:19:47.334 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-24T17:19:47.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCz3Z1p3p4nBxAAbW2FrXHu8UpPVHiUgcjYTA== --osd-uuid 9bfc0c25-4ce2-42c0-8877-0fe11dc1969f 2026-02-24T17:19:47.351 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:47.360+0000 7f3691ef8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:47.353 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:47.360+0000 7f3691ef8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:47.354 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:47.364+0000 7f3691ef8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:47.354 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:47.364+0000 7f3691ef8a40 -1 bdev(0x559ae2330400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-24T17:19:47.355 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:47.364+0000 7f3691ef8a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-24T17:19:48.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-24T17:19:48.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:19:48.037 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-24T17:19:48.038 INFO:tasks.workunit.client.0.trial066.stdout:adding osd5 key to auth repository 2026-02-24T17:19:48.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:19:48.335 INFO:tasks.workunit.client.0.trial066.stdout:start osd.5 2026-02-24T17:19:48.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-24T17:19:48.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:48.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:19:48.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:19:48.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:19:48.352 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:48.360+0000 7f3465335a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:48.353 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:48.360+0000 7f3465335a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:48.354 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:48.364+0000 7f3465335a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:48.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-24T17:19:48.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:48.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-24T17:19:48.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:48.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:48.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:48.550 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:48.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:48.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:48.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:19:48.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:48.775 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:48.784+0000 7f3465335a40 -1 Falling back to public interface 2026-02-24T17:19:49.423 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:49.432+0000 7f3465335a40 -1 osd.5 0 log_to_monitors true 2026-02-24T17:19:49.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:49.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:49.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:49.762 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:49.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:49.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:19:49.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:50.972 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:50.972 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:50.972 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:19:50.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:19:50.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:50.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:19:51.178 INFO:tasks.workunit.client.0.trial066.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2915691733,v1:127.0.0.1:6843/2915691733] [v2:127.0.0.1:6844/2915691733,v1:127.0.0.1:6845/2915691733] exists,up 9bfc0c25-4ce2-42c0-8877-0fe11dc1969f 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:19:51.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:19:51.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:19:51.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:19:51.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:19:51.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:19:51.181 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:19:51.181 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:51.181 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-24T17:19:51.182 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:19:51.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=256236f4-e9a7-4228-8001-2b366c53c302 2026-02-24T17:19:51.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 256236f4-e9a7-4228-8001-2b366c53c302' 2026-02-24T17:19:51.183 INFO:tasks.workunit.client.0.trial066.stdout:add osd6 256236f4-e9a7-4228-8001-2b366c53c302 2026-02-24T17:19:51.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:19:51.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC33Z1pD55GDBAA8mmMetXxJ3qv2otECAIXFA== 2026-02-24T17:19:51.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC33Z1pD55GDBAA8mmMetXxJ3qv2otECAIXFA=="}' 2026-02-24T17:19:51.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 256236f4-e9a7-4228-8001-2b366c53c302 -i td/test-erasure-eio/6/new.json 2026-02-24T17:19:51.413 INFO:tasks.workunit.client.0.trial066.stdout:6 2026-02-24T17:19:51.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-24T17:19:51.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC33Z1pD55GDBAA8mmMetXxJ3qv2otECAIXFA== --osd-uuid 256236f4-e9a7-4228-8001-2b366c53c302 2026-02-24T17:19:51.440 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:51.448+0000 7fa419938a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:51.441 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:51.448+0000 7fa419938a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:51.442 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:51.452+0000 7fa419938a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:51.443 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:51.452+0000 7fa419938a40 -1 bdev(0x564c6869a400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-24T17:19:51.443 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:51.452+0000 7fa419938a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-24T17:19:52.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-24T17:19:52.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:19:52.113 INFO:tasks.workunit.client.0.trial066.stdout:adding osd6 key to auth repository 2026-02-24T17:19:52.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-24T17:19:52.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:19:52.401 INFO:tasks.workunit.client.0.trial066.stdout:start osd.6 2026-02-24T17:19:52.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-24T17:19:52.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:19:52.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:19:52.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:19:52.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:19:52.426 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:52.436+0000 7ff3514f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:52.428 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:52.436+0000 7ff3514f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:52.429 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:52.436+0000 7ff3514f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:19:52.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-24T17:19:52.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:19:52.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-24T17:19:52.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:19:52.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:19:52.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:52.613 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:19:52.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:19:52.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:52.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:19:52.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:52.931 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:52.940+0000 7ff3514f1a40 -1 Falling back to public interface 2026-02-24T17:19:53.607 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:19:53.616+0000 7ff3514f1a40 -1 osd.6 0 log_to_monitors true 2026-02-24T17:19:53.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:53.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:53.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:19:53.823 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:19:53.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:53.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:19:54.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:19:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:19:55.034 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:19:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:19:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:19:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:19:55.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:19:55.249 INFO:tasks.workunit.client.0.trial066.stdout:osd.6 up in weight 1 up_from 50 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2347317626,v1:127.0.0.1:6851/2347317626] [v2:127.0.0.1:6852/2347317626,v1:127.0.0.1:6853/2347317626] exists,up 256236f4-e9a7-4228-8001-2b366c53c302 2026-02-24T17:19:55.249 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:19:55.249 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:19:55.249 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:19:55.250 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:19:55.250 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:19:55.250 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:19:55.250 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:19:55.250 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:19:55.250 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:19:55.250 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:19:55.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:19:55.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:19:55.307 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:19:55.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:19:55.316 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:19:30.956+0000 7f39447bea40 0 load: jerasure load: lrc 2026-02-24T17:19:55.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-24T17:19:55.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-24T17:19:55.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:19:55.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:19:55.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-24T17:19:55.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:19:55.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-24T17:19:55.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:19:55.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-24T17:19:55.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:19:55.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:19:55.965 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:19:55.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:19:56.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:19:56.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:19:56.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:19:56.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:19:56.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:19:56.978 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:19:56.978 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:19:56.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:19:56.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:19:56.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:19:57.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:19:57.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:19:57.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:19:57.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:19:57.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:19:57.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:19:57.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:19:57.299 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:19:57.299 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:19:57.299 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:19:57.299 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:19:57.299 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:19:57.300 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:19:57.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:19:57.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:57.300 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:19:57.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803783 2026-02-24T17:19:57.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803783 2026-02-24T17:19:57.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783' 2026-02-24T17:19:57.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:57.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:19:57.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607559 2026-02-24T17:19:57.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607559 2026-02-24T17:19:57.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559' 2026-02-24T17:19:57.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:57.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:19:57.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411334 2026-02-24T17:19:57.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411334 2026-02-24T17:19:57.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334' 2026-02-24T17:19:57.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:57.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:19:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149702 2026-02-24T17:19:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149702 2026-02-24T17:19:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702' 2026-02-24T17:19:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:57.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:19:57.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822661 2026-02-24T17:19:57.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822661 2026-02-24T17:19:57.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-154618822661' 2026-02-24T17:19:57.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:57.700 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:19:57.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593732 2026-02-24T17:19:57.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593732 2026-02-24T17:19:57.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-154618822661 5-184683593732' 2026-02-24T17:19:57.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:19:57.780 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:19:57.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364803 2026-02-24T17:19:57.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364803 2026-02-24T17:19:57.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-154618822661 5-184683593732 6-214748364803' 2026-02-24T17:19:57.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:57.857 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803783 2026-02-24T17:19:57.857 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:57.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:19:57.859 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803783 2026-02-24T17:19:57.859 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:57.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803783 2026-02-24T17:19:57.860 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803783 2026-02-24T17:19:57.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-24T17:19:57.861 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:19:58.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-24T17:19:58.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:19:59.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:19:59.081 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:19:59.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803783 2026-02-24T17:19:59.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:59.301 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607559 2026-02-24T17:19:59.301 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:59.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:19:59.303 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607559 2026-02-24T17:19:59.303 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:59.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607559 2026-02-24T17:19:59.305 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607559 2026-02-24T17:19:59.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-24T17:19:59.305 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:19:59.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-24T17:19:59.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:59.524 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411334 2026-02-24T17:19:59.525 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:59.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:19:59.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411334 2026-02-24T17:19:59.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:59.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411334 2026-02-24T17:19:59.528 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411334 2026-02-24T17:19:59.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-24T17:19:59.529 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:19:59.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-24T17:19:59.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:59.755 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149702 2026-02-24T17:19:59.755 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:59.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:19:59.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149702 2026-02-24T17:19:59.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:59.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149702 2026-02-24T17:19:59.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-24T17:19:59.758 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 111669149702 2026-02-24T17:19:59.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:19:59.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-24T17:19:59.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:19:59.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822661 2026-02-24T17:19:59.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:19:59.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:19:59.971 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822661 2026-02-24T17:19:59.972 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:19:59.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822661 2026-02-24T17:19:59.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822661' 2026-02-24T17:19:59.973 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 154618822661 2026-02-24T17:19:59.973 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:20:00.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822661 -lt 154618822661 2026-02-24T17:20:00.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:00.201 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-184683593732 2026-02-24T17:20:00.202 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:00.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:20:00.203 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-184683593732 2026-02-24T17:20:00.203 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:00.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593732 2026-02-24T17:20:00.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-24T17:20:00.205 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 184683593732 2026-02-24T17:20:00.205 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:20:00.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-24T17:20:00.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:00.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-214748364803 2026-02-24T17:20:00.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:00.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:20:00.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-214748364803 2026-02-24T17:20:00.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:00.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364803 2026-02-24T17:20:00.429 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 214748364803 2026-02-24T17:20:00.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 214748364803' 2026-02-24T17:20:00.429 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:20:00.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364804 -lt 214748364803 2026-02-24T17:20:00.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:20:00.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:20:00.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:20:00.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:20:01.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:20:01.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:20:01.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:20:01.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:20:01.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-24T17:20:01.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-24T17:20:01.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-24T17:20:01.642 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:20:01.642 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:20:01.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:20:01.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:20:01.642 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:20:01.950 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:20:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=mdata 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:20:02.150 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:20:02.151 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:20:02.151 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:20:02.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-24T17:20:02.373 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:02.373 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=5 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:20:02.374 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/5/type 2026-02-24T17:20:02.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:20:02.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-24T17:20:02.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:20:02.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-24T17:20:02.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:20:02.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:20:02.376 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:20:02.376 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-24T17:20:02.377 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-24T17:20:02.377 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-24T17:20:02.377 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:02.377 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:02.377 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:02.377 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.5.asok 2026-02-24T17:20:02.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:20:02.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:20:02.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:20:02.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:20:02.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.5 2026-02-24T17:20:02.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-24T17:20:02.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-24T17:20:02.440 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:02.440 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:02.440 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:02.441 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.5.asok 2026-02-24T17:20:02.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:20:02.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:20:02.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:02.502 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:20:02.502 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:02.502 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:02.502 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:02.502 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:02.502 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:02.502 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:20:02.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:20:02.504 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:20:02.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T17:20:02.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:02.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-24T17:20:02.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T17:20:02.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:20:02.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:20:02.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:20:02.528 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:02.536+0000 7f22e9f79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:02.529 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:02.536+0000 7f22e9f79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:02.531 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:02.540+0000 7f22e9f79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:02.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T17:20:02.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:02.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:20:02.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:02.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:02.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:02.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:02.715 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:02.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:02.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:20:02.895 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:02.904+0000 7f22e9f79a40 -1 Falling back to public interface 2026-02-24T17:20:02.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:03.623 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:03.632+0000 7f22e9f79a40 -1 osd.3 57 log_to_monitors true 2026-02-24T17:20:03.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:03.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:03.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:03.940 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:03.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:03.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 61 up_thru 61 down_at 58 last_clean_interval [26,57) [v2:127.0.0.1:6826/2879786391,v1:127.0.0.1:6827/2879786391] [v2:127.0.0.1:6828/2879786391,v1:127.0.0.1:6829/2879786391] exists,up 719494eb-c7c7-426a-a58d-f03a223497ff 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:20:04.156 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:20:04.157 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:20:04.157 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:20:04.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:20:04.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:20:04.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:20:04.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:20:04.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:20:04.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:20:04.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:20:04.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:20:04.260 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:20:04.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:20:04.478 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:04.478 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:20:04.478 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:20:04.478 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:20:04.479 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:04.479 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:20:04.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:20:04.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:04.479 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:20:04.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803787 2026-02-24T17:20:04.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803787 2026-02-24T17:20:04.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787' 2026-02-24T17:20:04.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:04.560 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:20:04.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607562 2026-02-24T17:20:04.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607562 2026-02-24T17:20:04.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562' 2026-02-24T17:20:04.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:04.641 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:20:04.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411337 2026-02-24T17:20:04.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411337 2026-02-24T17:20:04.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337' 2026-02-24T17:20:04.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:04.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:20:04.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=261993005059 2026-02-24T17:20:04.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 261993005059 2026-02-24T17:20:04.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-261993005059' 2026-02-24T17:20:04.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:04.804 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:20:04.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822664 2026-02-24T17:20:04.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822664 2026-02-24T17:20:04.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-261993005059 4-154618822664' 2026-02-24T17:20:04.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:04.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:20:04.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593735 2026-02-24T17:20:04.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593735 2026-02-24T17:20:04.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-261993005059 4-154618822664 5-184683593735' 2026-02-24T17:20:04.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:04.963 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:20:05.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=214748364807 2026-02-24T17:20:05.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 214748364807 2026-02-24T17:20:05.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-261993005059 4-154618822664 5-184683593735 6-214748364807' 2026-02-24T17:20:05.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:05.042 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803787 2026-02-24T17:20:05.042 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:05.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:20:05.043 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803787 2026-02-24T17:20:05.043 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:05.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803787 2026-02-24T17:20:05.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-24T17:20:05.045 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803787 2026-02-24T17:20:05.045 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:20:05.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-24T17:20:05.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:20:06.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:20:06.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:20:06.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-24T17:20:06.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:20:07.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:20:07.483 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:20:07.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-24T17:20:07.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:07.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607562 2026-02-24T17:20:07.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:07.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:20:07.699 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607562 2026-02-24T17:20:07.699 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:07.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607562 2026-02-24T17:20:07.700 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607562 2026-02-24T17:20:07.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607562' 2026-02-24T17:20:07.701 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:20:07.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607562 -lt 51539607562 2026-02-24T17:20:07.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:07.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411337 2026-02-24T17:20:07.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:07.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:20:07.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411337 2026-02-24T17:20:07.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:07.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411337 2026-02-24T17:20:07.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-24T17:20:07.931 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411337 2026-02-24T17:20:07.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:20:08.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411338 -lt 77309411337 2026-02-24T17:20:08.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:08.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-261993005059 2026-02-24T17:20:08.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:08.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:20:08.142 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-261993005059 2026-02-24T17:20:08.142 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:08.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=261993005059 2026-02-24T17:20:08.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 261993005059' 2026-02-24T17:20:08.143 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 261993005059 2026-02-24T17:20:08.144 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:20:08.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 261993005059 -lt 261993005059 2026-02-24T17:20:08.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:08.367 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822664 2026-02-24T17:20:08.367 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:08.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:20:08.369 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822664 2026-02-24T17:20:08.369 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:08.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822664 2026-02-24T17:20:08.370 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 154618822664 2026-02-24T17:20:08.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822664' 2026-02-24T17:20:08.371 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:20:08.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822665 -lt 154618822664 2026-02-24T17:20:08.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:08.579 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-184683593735 2026-02-24T17:20:08.579 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:08.580 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:20:08.581 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-184683593735 2026-02-24T17:20:08.581 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:08.582 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593735 2026-02-24T17:20:08.582 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 184683593735 2026-02-24T17:20:08.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-24T17:20:08.583 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:20:08.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593736 -lt 184683593735 2026-02-24T17:20:08.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:08.802 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-214748364807 2026-02-24T17:20:08.802 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:08.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:20:08.804 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-214748364807 2026-02-24T17:20:08.804 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:08.806 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=214748364807 2026-02-24T17:20:08.806 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 214748364807 2026-02-24T17:20:08.806 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 214748364807' 2026-02-24T17:20:08.806 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:20:09.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 214748364807 -lt 214748364807 2026-02-24T17:20:09.024 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:20:09.024 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:20:09.024 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:20:09.299 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:20:09.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:20:09.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:20:09.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:20:09.521 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:20:09.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:20:09.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:20:09.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:20:09.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:20:09.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:20:09.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:20:09.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-24T17:20:09.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:20:09.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:20:09.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-24T17:20:09.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:20:09.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:20:09.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:20:09.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:20:09.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:20:10.087 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:20:10.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:20:10.372 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:20:10.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:20:10.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:20:10.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:20:10.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:20:10.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:20:10.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:20:10.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:20:10.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:20:10.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:20:10.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:20:10.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:20:10.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:20:10.496 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:20:10.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:20:10.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:20:10.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:20:10.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:20:10.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:20:10.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:20:10.499 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:20:10.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:20:10.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:20:10.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:20:10.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:20:10.524 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:10.524 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:10.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:20:10.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:20:10.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:20:10.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:20:10.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:20:10.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:20:10.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:20:10.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:20:10.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:20:10.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:20:10.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:20:10.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:20:10.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:20:10.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:20:10.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:20:10.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:20:10.528 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:20:10.529 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:20:10.530 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:20:10.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:20:10.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:20:10.531 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:20:10.532 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:20:10.532 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:20:10.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:20:10.533 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:20:10.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:20:10.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:20:10.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:20:10.537 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:20:10.537 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:10.537 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:10.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:20:10.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:20:10.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:20:10.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:20:10.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:20:10.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:10.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:10.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:20:10.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:20:10.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:20:10.565 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:20:10.565 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:10.565 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:10.566 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:10.566 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:10.566 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:10.566 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:10.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:20:10.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:20:10.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:20:10.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:20:10.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:20:10.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:20:10.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:20:10.593 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:20:10.593 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:20:10.593 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:20:10.593 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:10.593 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:10.593 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:10.593 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:20:10.594 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:20:10.594 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:20:10.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:20:10.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:20:10.650 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:10.651 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:10.652 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:20:10.652 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:20:10.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:20:10.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:20:10.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:20:10.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:20:10.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:20:10.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:20:10.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:20:10.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:20:10.824 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:20:10.824 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:10.824 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:10.824 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:10.825 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:10.825 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:10.825 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:10.825 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:20:10.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:20:10.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:20:10.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:20:10.953 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:20:10.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:20:11.963 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:20:11.963 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:20:11.963 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:20:11.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:11.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:11.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:11.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:20:11.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:20:11.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:20:12.013 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:20:12.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:20:12.020 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:20:10.592+0000 7f9c81a93ec0 0 load: jerasure load: lrc 2026-02-24T17:20:12.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-24T17:20:12.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-24T17:20:12.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-24T17:20:12.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-24T17:20:12.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-24T17:20:12.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:20:12.022 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-24T17:20:12.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-24T17:20:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:20:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:20:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:12.028 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6ba8d177-2925-4bc7-b8f4-f97f1fe699d5 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6ba8d177-2925-4bc7-b8f4-f97f1fe699d5' 2026-02-24T17:20:12.029 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 6ba8d177-2925-4bc7-b8f4-f97f1fe699d5 2026-02-24T17:20:12.030 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:12.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDM3Z1p16brAhAAajH0BuSReL8cvj9L0eRktg== 2026-02-24T17:20:12.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDM3Z1p16brAhAAajH0BuSReL8cvj9L0eRktg=="}' 2026-02-24T17:20:12.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6ba8d177-2925-4bc7-b8f4-f97f1fe699d5 -i td/test-erasure-eio/0/new.json 2026-02-24T17:20:12.150 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:12.157 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:20:12.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDM3Z1p16brAhAAajH0BuSReL8cvj9L0eRktg== --osd-uuid 6ba8d177-2925-4bc7-b8f4-f97f1fe699d5 2026-02-24T17:20:12.174 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:12.184+0000 7ff4770e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:12.176 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:12.184+0000 7ff4770e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:12.176 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:12.184+0000 7ff4770e0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:12.177 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:12.184+0000 7ff4770e0a40 -1 bdev(0x55fdfce0e400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:12.177 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:12.184+0000 7ff4770e0a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:20:12.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:20:12.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:12.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:20:12.862 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:20:12.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:13.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:20:13.016 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:20:13.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:13.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:13.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:13.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:13.037 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:13.044+0000 7f1711ddda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:13.038 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:13.048+0000 7f1711ddda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:13.039 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:13.048+0000 7f1711ddda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:13.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:20:13.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:13.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:20:13.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:13.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:13.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:13.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:13.132 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:13.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:13.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:20:13.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:13.455 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:13.464+0000 7f1711ddda40 -1 Falling back to public interface 2026-02-24T17:20:14.015 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:14.024+0000 7f1711ddda40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:20:14.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:14.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:14.351 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:14.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:14.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:14.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:20:14.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:15.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:15.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:15.574 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:15.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:15.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:15.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:20:15.789 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2297397427,v1:127.0.0.1:6803/2297397427] [v2:127.0.0.1:6804/2297397427,v1:127.0.0.1:6805/2297397427] exists,up 6ba8d177-2925-4bc7-b8f4-f97f1fe699d5 2026-02-24T17:20:15.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:20:15.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:15.794 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:15.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:15.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:15.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:15.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:15.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:20:15.796 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:15.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ee911a54-0ad0-488e-89c2-d22fccbefca5 2026-02-24T17:20:15.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 ee911a54-0ad0-488e-89c2-d22fccbefca5' 2026-02-24T17:20:15.797 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 ee911a54-0ad0-488e-89c2-d22fccbefca5 2026-02-24T17:20:15.797 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:15.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDP3Z1pFPKXMBAARk5SPdAZ49uVJOIyHm4JjA== 2026-02-24T17:20:15.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDP3Z1pFPKXMBAARk5SPdAZ49uVJOIyHm4JjA=="}' 2026-02-24T17:20:15.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ee911a54-0ad0-488e-89c2-d22fccbefca5 -i td/test-erasure-eio/1/new.json 2026-02-24T17:20:16.027 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:16.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:20:16.037 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDP3Z1pFPKXMBAARk5SPdAZ49uVJOIyHm4JjA== --osd-uuid ee911a54-0ad0-488e-89c2-d22fccbefca5 2026-02-24T17:20:16.054 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:16.064+0000 7f73d56d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:16.056 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:16.064+0000 7f73d56d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:16.057 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:16.064+0000 7f73d56d8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:16.058 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:16.064+0000 7f73d56d8a40 -1 bdev(0x5615a4efe400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:16.058 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:16.064+0000 7f73d56d8a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:20:16.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:20:16.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:16.706 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:20:16.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:20:16.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:17.005 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:20:17.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:20:17.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:17.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:17.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:17.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:17.029 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:17.036+0000 7f8b2daa9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:17.031 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:17.040+0000 7f8b2daa9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:17.032 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:17.040+0000 7f8b2daa9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:17.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:17.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:20:17.432 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:17.440+0000 7f8b2daa9a40 -1 Falling back to public interface 2026-02-24T17:20:17.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:18.128 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:18.136+0000 7f8b2daa9a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:20:18.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:18.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:18.436 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:18.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:18.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:18.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:20:18.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:19.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:19.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:19.667 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:19.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:19.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:19.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:20:19.878 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3560156720,v1:127.0.0.1:6811/3560156720] [v2:127.0.0.1:6812/3560156720,v1:127.0.0.1:6813/3560156720] exists,up ee911a54-0ad0-488e-89c2-d22fccbefca5 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:19.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:19.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:19.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:19.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:19.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:19.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:19.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e8738100-b470-4ace-b1ce-beb4607a06cf 2026-02-24T17:20:19.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 e8738100-b470-4ace-b1ce-beb4607a06cf' 2026-02-24T17:20:19.884 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 e8738100-b470-4ace-b1ce-beb4607a06cf 2026-02-24T17:20:19.884 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:19.895 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDT3Z1pKs36NRAA6vGO2elgEE/wpP1B5SWioQ== 2026-02-24T17:20:19.895 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDT3Z1pKs36NRAA6vGO2elgEE/wpP1B5SWioQ=="}' 2026-02-24T17:20:19.896 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e8738100-b470-4ace-b1ce-beb4607a06cf -i td/test-erasure-eio/2/new.json 2026-02-24T17:20:20.117 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:20.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:20:20.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDT3Z1pKs36NRAA6vGO2elgEE/wpP1B5SWioQ== --osd-uuid e8738100-b470-4ace-b1ce-beb4607a06cf 2026-02-24T17:20:20.144 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:20.152+0000 7fdca4c9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:20.146 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:20.152+0000 7fdca4c9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:20.147 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:20.156+0000 7fdca4c9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:20.147 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:20.156+0000 7fdca4c9da40 -1 bdev(0x562279d4e400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:20.148 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:20.156+0000 7fdca4c9da40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:20:20.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:20:20.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:20.830 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:20:20.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:20:20.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:21.128 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:20:21.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:20:21.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:21.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:21.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:21.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:21.153 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:21.160+0000 7f583bc18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:21.155 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:21.164+0000 7f583bc18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:21.156 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:21.164+0000 7f583bc18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:21.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:20:21.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:21.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:20:21.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:21.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:21.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:21.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:21.339 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:21.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:21.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:20:21.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:21.583 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:21.592+0000 7f583bc18a40 -1 Falling back to public interface 2026-02-24T17:20:22.436 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:22.444+0000 7f583bc18a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:20:22.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:22.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:22.547 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:22.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:22.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:22.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:20:22.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:23.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:23.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:23.770 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:23.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:23.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:23.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:20:23.985 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3647846299,v1:127.0.0.1:6819/3647846299] [v2:127.0.0.1:6820/3647846299,v1:127.0.0.1:6821/3647846299] exists,up e8738100-b470-4ace-b1ce-beb4607a06cf 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:23.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:23.987 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:23.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:20:23.990 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:23.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2f479cc3-7c1f-45ac-b0ce-0669a7e84ab0 2026-02-24T17:20:23.991 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 2f479cc3-7c1f-45ac-b0ce-0669a7e84ab0 2026-02-24T17:20:23.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2f479cc3-7c1f-45ac-b0ce-0669a7e84ab0' 2026-02-24T17:20:23.991 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:24.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDY3Z1paTLbABAAQzFn0ai9TwzECPwOQAr8eA== 2026-02-24T17:20:24.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDY3Z1paTLbABAAQzFn0ai9TwzECPwOQAr8eA=="}' 2026-02-24T17:20:24.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2f479cc3-7c1f-45ac-b0ce-0669a7e84ab0 -i td/test-erasure-eio/3/new.json 2026-02-24T17:20:24.232 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:20:24.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:20:24.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDY3Z1paTLbABAAQzFn0ai9TwzECPwOQAr8eA== --osd-uuid 2f479cc3-7c1f-45ac-b0ce-0669a7e84ab0 2026-02-24T17:20:24.260 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:24.267+0000 7fc3a2675a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:24.261 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:24.267+0000 7fc3a2675a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:24.262 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:24.271+0000 7fc3a2675a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:24.263 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:24.271+0000 7fc3a2675a40 -1 bdev(0x55d3f0630400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:24.263 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:24.271+0000 7fc3a2675a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:20:24.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:20:24.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:24.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:20:24.918 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:20:24.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:25.214 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:20:25.215 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:20:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:25.237 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:25.243+0000 7f301e89aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:25.239 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:25.247+0000 7f301e89aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:25.240 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:25.247+0000 7f301e89aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:25.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:20:25.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:25.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:20:25.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:25.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:25.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:25.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:25.428 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:25.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:25.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:20:25.628 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:25.635+0000 7f301e89aa40 -1 Falling back to public interface 2026-02-24T17:20:25.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:26.285 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:26.291+0000 7f301e89aa40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:20:26.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:26.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:26.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:26.651 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:26.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:26.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:20:26.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:27.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:27.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:27.876 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:27.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:27.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:27.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/120125937,v1:127.0.0.1:6827/120125937] [v2:127.0.0.1:6828/120125937,v1:127.0.0.1:6829/120125937] exists,up 2f479cc3-7c1f-45ac-b0ce-0669a7e84ab0 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:28.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:28.091 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:28.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-24T17:20:28.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:28.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=088a858e-a684-4c7f-bf41-a283e64adb2c 2026-02-24T17:20:28.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 088a858e-a684-4c7f-bf41-a283e64adb2c' 2026-02-24T17:20:28.094 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 088a858e-a684-4c7f-bf41-a283e64adb2c 2026-02-24T17:20:28.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:28.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDc3Z1pidoGBxAAPtOiBC/Vp8eRWD2am5AGEg== 2026-02-24T17:20:28.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDc3Z1pidoGBxAAPtOiBC/Vp8eRWD2am5AGEg=="}' 2026-02-24T17:20:28.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 088a858e-a684-4c7f-bf41-a283e64adb2c -i td/test-erasure-eio/4/new.json 2026-02-24T17:20:28.334 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:20:28.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-24T17:20:28.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDc3Z1pidoGBxAAPtOiBC/Vp8eRWD2am5AGEg== --osd-uuid 088a858e-a684-4c7f-bf41-a283e64adb2c 2026-02-24T17:20:28.363 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:28.371+0000 7f45852fda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:28.364 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:28.371+0000 7f45852fda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:28.365 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:28.371+0000 7f45852fda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:28.366 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:28.375+0000 7f45852fda40 -1 bdev(0x55a362af0400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:28.366 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:28.375+0000 7f45852fda40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-24T17:20:29.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-24T17:20:29.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:29.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:20:29.074 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:20:29.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:29.364 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:20:29.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:20:29.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:29.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:29.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:29.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:29.387 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:29.395+0000 7f3adb1b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:29.389 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:29.395+0000 7f3adb1b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:29.390 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:29.399+0000 7f3adb1b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:29.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:20:29.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:29.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:20:29.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:29.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:29.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:29.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:29.577 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:29.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:29.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:20:29.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:29.880 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:29.887+0000 7f3adb1b0a40 -1 Falling back to public interface 2026-02-24T17:20:30.532 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:30.539+0000 7f3adb1b0a40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:20:30.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:30.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:30.795 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:30.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:30.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:30.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:20:31.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:32.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:32.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:32.014 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:32.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:32.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:32.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:20:32.229 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 36 up_thru 39 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/59968676,v1:127.0.0.1:6835/59968676] [v2:127.0.0.1:6836/59968676,v1:127.0.0.1:6837/59968676] exists,up 088a858e-a684-4c7f-bf41-a283e64adb2c 2026-02-24T17:20:32.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:32.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:32.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:32.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-24T17:20:32.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:32.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b47e5774-03c7-4851-8f67-0157f05a8ea6 2026-02-24T17:20:32.236 INFO:tasks.workunit.client.0.trial066.stdout:add osd5 b47e5774-03c7-4851-8f67-0157f05a8ea6 2026-02-24T17:20:32.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 b47e5774-03c7-4851-8f67-0157f05a8ea6' 2026-02-24T17:20:32.236 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:32.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDg3Z1plO0/DxAAC/AFlbLqlAWvfBX5ybrptg== 2026-02-24T17:20:32.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDg3Z1plO0/DxAAC/AFlbLqlAWvfBX5ybrptg=="}' 2026-02-24T17:20:32.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b47e5774-03c7-4851-8f67-0157f05a8ea6 -i td/test-erasure-eio/5/new.json 2026-02-24T17:20:32.462 INFO:tasks.workunit.client.0.trial066.stdout:5 2026-02-24T17:20:32.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-24T17:20:32.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDg3Z1plO0/DxAAC/AFlbLqlAWvfBX5ybrptg== --osd-uuid b47e5774-03c7-4851-8f67-0157f05a8ea6 2026-02-24T17:20:32.497 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:32.503+0000 7f83636c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:32.499 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:32.507+0000 7f83636c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:32.500 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:32.507+0000 7f83636c7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:32.501 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:32.507+0000 7f83636c7a40 -1 bdev(0x561e92e8c400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:32.501 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:32.507+0000 7f83636c7a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-24T17:20:33.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-24T17:20:33.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:33.134 INFO:tasks.workunit.client.0.trial066.stdout:adding osd5 key to auth repository 2026-02-24T17:20:33.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-24T17:20:33.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:33.429 INFO:tasks.workunit.client.0.trial066.stdout:start osd.5 2026-02-24T17:20:33.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-24T17:20:33.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:33.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:33.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:33.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:33.455 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:33.463+0000 7fded22c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:33.457 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:33.463+0000 7fded22c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:33.458 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:33.463+0000 7fded22c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:33.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-24T17:20:33.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:33.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-24T17:20:33.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:33.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:33.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:33.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:33.640 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:33.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:33.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:20:33.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:33.912 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:33.919+0000 7fded22c6a40 -1 Falling back to public interface 2026-02-24T17:20:34.589 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:34.595+0000 7fded22c6a40 -1 osd.5 0 log_to_monitors true 2026-02-24T17:20:34.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:34.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:34.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:34.850 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:34.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:34.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:20:35.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:36.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:36.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:36.069 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:36.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:36.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:36.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:20:36.285 INFO:tasks.workunit.client.0.trial066.stdout:osd.5 up in weight 1 up_from 46 up_thru 47 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2346721391,v1:127.0.0.1:6843/2346721391] [v2:127.0.0.1:6844/2346721391,v1:127.0.0.1:6845/2346721391] exists,up b47e5774-03c7-4851-8f67-0157f05a8ea6 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:36.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:36.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-24T17:20:36.288 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:36.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=91947f74-251b-4ca1-b56d-19d010fae36a 2026-02-24T17:20:36.289 INFO:tasks.workunit.client.0.trial066.stdout:add osd6 91947f74-251b-4ca1-b56d-19d010fae36a 2026-02-24T17:20:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 91947f74-251b-4ca1-b56d-19d010fae36a' 2026-02-24T17:20:36.290 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:36.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDk3Z1pRMeXEhAAFsPeXQHxB4omE6QQNQ1Klg== 2026-02-24T17:20:36.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDk3Z1pRMeXEhAAFsPeXQHxB4omE6QQNQ1Klg=="}' 2026-02-24T17:20:36.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 91947f74-251b-4ca1-b56d-19d010fae36a -i td/test-erasure-eio/6/new.json 2026-02-24T17:20:36.522 INFO:tasks.workunit.client.0.trial066.stdout:6 2026-02-24T17:20:36.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-24T17:20:36.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDk3Z1pRMeXEhAAFsPeXQHxB4omE6QQNQ1Klg== --osd-uuid 91947f74-251b-4ca1-b56d-19d010fae36a 2026-02-24T17:20:36.552 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:36.559+0000 7fa1c8607a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:36.553 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:36.559+0000 7fa1c8607a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:36.554 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:36.563+0000 7fa1c8607a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:36.554 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:36.563+0000 7fa1c8607a40 -1 bdev(0x562d8580c400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:36.555 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:36.563+0000 7fa1c8607a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-24T17:20:37.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-24T17:20:37.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:37.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-24T17:20:37.607 INFO:tasks.workunit.client.0.trial066.stdout:adding osd6 key to auth repository 2026-02-24T17:20:37.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:37.906 INFO:tasks.workunit.client.0.trial066.stdout:start osd.6 2026-02-24T17:20:37.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-24T17:20:37.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:37.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:37.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:37.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:37.933 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:37.939+0000 7f71633bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:37.935 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:37.943+0000 7f71633bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:37.936 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:37.943+0000 7f71633bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:38.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:20:38.331 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:38.339+0000 7f71633bba40 -1 Falling back to public interface 2026-02-24T17:20:38.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:39.009 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:39.015+0000 7f71633bba40 -1 osd.6 0 log_to_monitors true 2026-02-24T17:20:39.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:39.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:39.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:39.338 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:39.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:39.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:20:39.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:40.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:40.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:40.551 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:40.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:40.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:40.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:20:40.767 INFO:tasks.workunit.client.0.trial066.stdout:osd.6 up in weight 1 up_from 54 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3343815139,v1:127.0.0.1:6851/3343815139] [v2:127.0.0.1:6852/3343815139,v1:127.0.0.1:6853/3343815139] exists,up 91947f74-251b-4ca1-b56d-19d010fae36a 2026-02-24T17:20:40.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:40.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:40.769 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:40.770 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:40.770 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:40.770 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:40.770 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:40.770 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-24T17:20:40.771 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:40.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=750d2a09-904e-4dc8-9ca5-bbe759cf849a 2026-02-24T17:20:40.772 INFO:tasks.workunit.client.0.trial066.stdout:add osd7 750d2a09-904e-4dc8-9ca5-bbe759cf849a 2026-02-24T17:20:40.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 750d2a09-904e-4dc8-9ca5-bbe759cf849a' 2026-02-24T17:20:40.772 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:40.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDo3Z1pyM5YLxAAt1VfmylRGA8f8nyQhFNlqg== 2026-02-24T17:20:40.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDo3Z1pyM5YLxAAt1VfmylRGA8f8nyQhFNlqg=="}' 2026-02-24T17:20:40.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 750d2a09-904e-4dc8-9ca5-bbe759cf849a -i td/test-erasure-eio/7/new.json 2026-02-24T17:20:41.000 INFO:tasks.workunit.client.0.trial066.stdout:7 2026-02-24T17:20:41.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-24T17:20:41.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDo3Z1pyM5YLxAAt1VfmylRGA8f8nyQhFNlqg== --osd-uuid 750d2a09-904e-4dc8-9ca5-bbe759cf849a 2026-02-24T17:20:41.035 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:41.043+0000 7f1c368b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:41.036 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:41.043+0000 7f1c368b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:41.037 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:41.043+0000 7f1c368b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:41.038 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:41.043+0000 7f1c368b9a40 -1 bdev(0x5646d7b02400 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:41.038 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:41.047+0000 7f1c368b9a40 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-24T17:20:41.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-24T17:20:41.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:41.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-24T17:20:41.711 INFO:tasks.workunit.client.0.trial066.stdout:adding osd7 key to auth repository 2026-02-24T17:20:41.711 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:42.012 INFO:tasks.workunit.client.0.trial066.stdout:start osd.7 2026-02-24T17:20:42.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-24T17:20:42.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:42.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:42.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:42.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:42.037 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:42.043+0000 7f0a5f199a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:42.039 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:42.047+0000 7f0a5f199a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:42.040 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:42.047+0000 7f0a5f199a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:42.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-24T17:20:42.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:42.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-24T17:20:42.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:42.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:42.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:42.220 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:42.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:42.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:42.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-24T17:20:42.423 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:42.431+0000 7f0a5f199a40 -1 Falling back to public interface 2026-02-24T17:20:42.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:43.075 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:43.083+0000 7f0a5f199a40 -1 osd.7 0 log_to_monitors true 2026-02-24T17:20:43.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:43.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:43.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:43.435 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:43.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-24T17:20:43.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:43.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:44.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:44.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:44.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:44.660 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:44.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:44.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-24T17:20:44.874 INFO:tasks.workunit.client.0.trial066.stdout:osd.7 up in weight 1 up_from 62 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/4078261412,v1:127.0.0.1:6859/4078261412] [v2:127.0.0.1:6860/4078261412,v1:127.0.0.1:6861/4078261412] exists,up 750d2a09-904e-4dc8-9ca5-bbe759cf849a 2026-02-24T17:20:44.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:44.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:44.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:44.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:20:44.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-24T17:20:44.877 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:20:44.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1fedfa08-3cc9-4603-93b9-679e762834e0 2026-02-24T17:20:44.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 1fedfa08-3cc9-4603-93b9-679e762834e0' 2026-02-24T17:20:44.878 INFO:tasks.workunit.client.0.trial066.stdout:add osd8 1fedfa08-3cc9-4603-93b9-679e762834e0 2026-02-24T17:20:44.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:20:44.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDs3Z1p25S7NRAA0wqEmHLlmLgMX3ChaaLEoQ== 2026-02-24T17:20:44.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDs3Z1p25S7NRAA0wqEmHLlmLgMX3ChaaLEoQ=="}' 2026-02-24T17:20:44.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1fedfa08-3cc9-4603-93b9-679e762834e0 -i td/test-erasure-eio/8/new.json 2026-02-24T17:20:45.115 INFO:tasks.workunit.client.0.trial066.stdout:8 2026-02-24T17:20:45.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-24T17:20:45.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDs3Z1p25S7NRAA0wqEmHLlmLgMX3ChaaLEoQ== --osd-uuid 1fedfa08-3cc9-4603-93b9-679e762834e0 2026-02-24T17:20:45.141 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:45.147+0000 7f375d8b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:45.143 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:45.151+0000 7f375d8b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:45.144 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:45.151+0000 7f375d8b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:45.145 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:45.151+0000 7f375d8b2a40 -1 bdev(0x557062358400 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-24T17:20:45.145 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:45.151+0000 7f375d8b2a40 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-24T17:20:45.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-24T17:20:45.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:20:45.790 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-24T17:20:45.790 INFO:tasks.workunit.client.0.trial066.stdout:adding osd8 key to auth repository 2026-02-24T17:20:45.790 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:20:46.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-24T17:20:46.099 INFO:tasks.workunit.client.0.trial066.stdout:start osd.8 2026-02-24T17:20:46.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:20:46.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:20:46.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:20:46.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:20:46.122 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:46.131+0000 7f5d49573a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:46.124 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:46.131+0000 7f5d49573a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:46.125 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:46.131+0000 7f5d49573a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:20:46.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-24T17:20:46.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:20:46.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-24T17:20:46.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:20:46.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:20:46.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:46.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:20:46.306 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:20:46.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:46.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-24T17:20:46.484 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:46.491+0000 7f5d49573a40 -1 Falling back to public interface 2026-02-24T17:20:46.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:47.107 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:20:47.115+0000 7f5d49573a40 -1 osd.8 0 log_to_monitors true 2026-02-24T17:20:47.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:47.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:47.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:20:47.526 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:20:47.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:47.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-24T17:20:47.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:20:48.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:20:48.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:20:48.753 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:20:48.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:20:48.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:20:48.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-24T17:20:48.967 INFO:tasks.workunit.client.0.trial066.stdout:osd.8 up in weight 1 up_from 71 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/4261952558,v1:127.0.0.1:6867/4261952558] [v2:127.0.0.1:6868/4261952558,v1:127.0.0.1:6869/4261952558] exists,up 1fedfa08-3cc9-4603-93b9-679e762834e0 2026-02-24T17:20:48.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:20:48.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:20:48.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:20:48.968 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:20:48.969 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:20:48.969 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:20:48.969 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:48.970 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:48.970 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:48.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:20:48.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:20:48.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:20:49.023 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:20:49.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:20:49.032 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:20:13.468+0000 7f1711ddda40 0 load: jerasure load: lrc 2026-02-24T17:20:49.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-24T17:20:49.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-24T17:20:49.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:20:49.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:20:49.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-24T17:20:49.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:20:49.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-24T17:20:49.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:20:49.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-24T17:20:49.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:20:49.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:20:49.700 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:20:49.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:20:50.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:20:50.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:20:50.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:20:50.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:20:50.712 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:20:50.713 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:20:50.713 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:20:50.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:20:50.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:20:50.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:20:50.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:20:50.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:20:50.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:20:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:20:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:20:50.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:20:51.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:20:51.024 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:51.024 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:8' 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:20:51.104 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803786 2026-02-24T17:20:51.104 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803786 2026-02-24T17:20:51.104 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786' 2026-02-24T17:20:51.104 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.104 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:20:51.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607561 2026-02-24T17:20:51.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607561 2026-02-24T17:20:51.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561' 2026-02-24T17:20:51.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.184 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:20:51.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411337 2026-02-24T17:20:51.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411337 2026-02-24T17:20:51.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337' 2026-02-24T17:20:51.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.263 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:20:51.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116999 2026-02-24T17:20:51.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116999 2026-02-24T17:20:51.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-115964116999' 2026-02-24T17:20:51.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:20:51.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822663 2026-02-24T17:20:51.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822663 2026-02-24T17:20:51.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-115964116999 4-154618822663' 2026-02-24T17:20:51.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.419 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:20:51.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=197568495622 2026-02-24T17:20:51.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 197568495622 2026-02-24T17:20:51.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-115964116999 4-154618822663 5-197568495622' 2026-02-24T17:20:51.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.501 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:20:51.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=231928233989 2026-02-24T17:20:51.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 231928233989 2026-02-24T17:20:51.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-115964116999 4-154618822663 5-197568495622 6-231928233989' 2026-02-24T17:20:51.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.578 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:20:51.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=266287972356 2026-02-24T17:20:51.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 266287972356 2026-02-24T17:20:51.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-115964116999 4-154618822663 5-197568495622 6-231928233989 7-266287972356' 2026-02-24T17:20:51.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:51.658 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:20:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=304942678019 2026-02-24T17:20:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 304942678019 2026-02-24T17:20:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411337 3-115964116999 4-154618822663 5-197568495622 6-231928233989 7-266287972356 8-304942678019' 2026-02-24T17:20:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:51.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803786 2026-02-24T17:20:51.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:51.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:20:51.743 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803786 2026-02-24T17:20:51.743 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:51.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803786 2026-02-24T17:20:51.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-24T17:20:51.745 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803786 2026-02-24T17:20:51.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:20:51.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-24T17:20:51.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:20:52.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:20:52.963 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:20:53.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-24T17:20:53.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:53.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607561 2026-02-24T17:20:53.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:53.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:20:53.187 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607561 2026-02-24T17:20:53.187 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:53.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607561 2026-02-24T17:20:53.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-24T17:20:53.189 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607561 2026-02-24T17:20:53.190 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:20:53.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-24T17:20:53.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:53.409 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411337 2026-02-24T17:20:53.409 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:53.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:20:53.410 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411337 2026-02-24T17:20:53.410 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:53.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411337 2026-02-24T17:20:53.412 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411337 2026-02-24T17:20:53.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-24T17:20:53.413 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:20:53.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-24T17:20:53.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:53.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964116999 2026-02-24T17:20:53.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:53.632 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:20:53.632 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964116999 2026-02-24T17:20:53.632 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:53.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116999 2026-02-24T17:20:53.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964116999' 2026-02-24T17:20:53.634 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 115964116999 2026-02-24T17:20:53.634 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:20:53.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117000 -lt 115964116999 2026-02-24T17:20:53.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:53.862 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822663 2026-02-24T17:20:53.863 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:53.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:20:53.864 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822663 2026-02-24T17:20:53.864 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:53.865 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822663 2026-02-24T17:20:53.866 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 154618822663 2026-02-24T17:20:53.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822663' 2026-02-24T17:20:53.866 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:20:54.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822664 -lt 154618822663 2026-02-24T17:20:54.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:54.096 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-197568495622 2026-02-24T17:20:54.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:54.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:20:54.099 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-197568495622 2026-02-24T17:20:54.099 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:54.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=197568495622 2026-02-24T17:20:54.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 197568495622' 2026-02-24T17:20:54.101 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 197568495622 2026-02-24T17:20:54.101 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:20:54.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 197568495622 -lt 197568495622 2026-02-24T17:20:54.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:54.323 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-231928233989 2026-02-24T17:20:54.324 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:54.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:20:54.325 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-231928233989 2026-02-24T17:20:54.326 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:54.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=231928233989 2026-02-24T17:20:54.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 231928233989' 2026-02-24T17:20:54.327 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 231928233989 2026-02-24T17:20:54.327 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:20:54.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 231928233989 -lt 231928233989 2026-02-24T17:20:54.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:54.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 7-266287972356 2026-02-24T17:20:54.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:54.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=7 2026-02-24T17:20:54.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 7-266287972356 2026-02-24T17:20:54.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:54.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=266287972356 2026-02-24T17:20:54.552 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.7 seq 266287972356 2026-02-24T17:20:54.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.7 seq 266287972356' 2026-02-24T17:20:54.552 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-24T17:20:54.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 266287972356 -lt 266287972356 2026-02-24T17:20:54.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:54.780 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-304942678019 2026-02-24T17:20:54.780 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:54.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:20:54.782 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-304942678019 2026-02-24T17:20:54.782 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:54.783 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=304942678019 2026-02-24T17:20:54.783 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 304942678019' 2026-02-24T17:20:54.783 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.8 seq 304942678019 2026-02-24T17:20:54.784 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:20:55.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 304942678019 -lt 304942678019 2026-02-24T17:20:55.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:20:55.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:20:55.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:20:55.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:20:55.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:20:55.288 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:20:55.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:20:55.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:20:55.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:20:55.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:20:55.289 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:20:55.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:20:55.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:20:55.506 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:20:55.506 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:20:55.781 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:20:55.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:20:55.803 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:20:55.804 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:20:55.804 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:20:56.016 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr:7' 2026-02-24T17:20:56.017 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-24T17:20:56.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-24T17:20:56.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:20:56.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-24T17:20:56.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:20:56.018 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-24T17:20:56.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:20:56.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-24T17:20:56.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:20:56.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-24T17:20:56.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:20:56.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:20:56.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:20:56.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:20:56.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:20:56.083 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-24T17:20:56.083 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:20:56.083 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:20:56.083 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:56.084 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:56.084 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:56.084 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:20:56.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:20:56.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:20:56.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=3 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:20:56.145 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:20:56.146 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:20:56.355 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:20:56.356 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:56.356 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:56.356 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:20:56.356 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:20:56.356 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:20:56.356 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:20:56.356 INFO:tasks.workunit.client.0.trial066.stderr:7' 2026-02-24T17:20:56.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-24T17:20:56.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-24T17:20:56.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:20:56.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=0 2026-02-24T17:20:56.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:20:56.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/0/type 2026-02-24T17:20:56.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:20:56.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:56.360 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:56.361 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:20:56.361 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:20:56.421 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:20:56.422 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.0 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:56.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:20:56.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:20:56.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-24T17:20:56.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-24T17:20:56.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:20:56.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:20:56.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:20:56.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:20:56.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=4 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:20:56.486 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:20:56.487 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:20:56.487 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:7' 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:20:56.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=6 2026-02-24T17:20:56.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:20:56.698 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/6/type 2026-02-24T17:20:56.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:20:56.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-24T17:20:56.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:20:56.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-24T17:20:56.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:20:56.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:20:56.699 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:20:56.699 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-24T17:20:56.699 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-24T17:20:56.699 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-24T17:20:56.700 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:56.700 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:56.700 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:56.700 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.6.asok 2026-02-24T17:20:56.700 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:20:56.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:20:56.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:20:56.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:20:56.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.6 2026-02-24T17:20:56.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-24T17:20:56.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-24T17:20:56.764 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:20:56.764 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:20:56.764 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:20:56.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.6.asok 2026-02-24T17:20:56.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:20:56.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-24T17:20:56.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-24T17:20:56.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:20:56.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:20:56.827 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:20:56.827 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:20:57.040 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:20:57.040 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:7' 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-24T17:20:57.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-24T17:20:57.042 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:20:57.042 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:20:57.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:20:57.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:20:57.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:20:57.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:20:57.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-24T17:20:57.562 INFO:tasks.workunit.client.0.trial066.stderr:osd.7 is already down. 2026-02-24T17:20:57.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-24T17:20:57.816 INFO:tasks.workunit.client.0.trial066.stderr:osd.7 is already out. 2026-02-24T17:20:57.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-24T17:20:57.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:20:57.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:20:57.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:20:57.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:20:57.826 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:20:57.826 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:20:57.827 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:20:57.827 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:20:57.827 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:20:57.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:20:57.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:20:57.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:20:57.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:20:57.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:20:57.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:6 2026-02-24T17:20:58.146 INFO:tasks.workunit.client.0.trial066.stderr:7 2026-02-24T17:20:58.147 INFO:tasks.workunit.client.0.trial066.stderr:8' 2026-02-24T17:20:58.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:20:58.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.147 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:20:58.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803789 2026-02-24T17:20:58.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803789 2026-02-24T17:20:58.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789' 2026-02-24T17:20:58.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.224 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:20:58.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607564 2026-02-24T17:20:58.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607564 2026-02-24T17:20:58.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-51539607564' 2026-02-24T17:20:58.306 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.306 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:20:58.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411340 2026-02-24T17:20:58.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411340 2026-02-24T17:20:58.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-51539607564 2-77309411340' 2026-02-24T17:20:58.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.385 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:20:58.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117003 2026-02-24T17:20:58.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117003 2026-02-24T17:20:58.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-51539607564 2-77309411340 3-115964117003' 2026-02-24T17:20:58.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.466 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:20:58.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822667 2026-02-24T17:20:58.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822667 2026-02-24T17:20:58.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-51539607564 2-77309411340 3-115964117003 4-154618822667' 2026-02-24T17:20:58.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:20:58.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=197568495625 2026-02-24T17:20:58.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 197568495625 2026-02-24T17:20:58.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-51539607564 2-77309411340 3-115964117003 4-154618822667 5-197568495625' 2026-02-24T17:20:58.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.624 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:20:58.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=231928233992 2026-02-24T17:20:58.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 231928233992 2026-02-24T17:20:58.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-51539607564 2-77309411340 3-115964117003 4-154618822667 5-197568495625 6-231928233992' 2026-02-24T17:20:58.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.7 flush_pg_stats 2026-02-24T17:20:58.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T17:20:58.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T17:20:58.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T17:20:58.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T17:20:58.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:20:58.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.8 flush_pg_stats 2026-02-24T17:20:58.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=304942678023 2026-02-24T17:20:58.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 304942678023 2026-02-24T17:20:58.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803789 1-51539607564 2-77309411340 3-115964117003 4-154618822667 5-197568495625 6-231928233992 8-304942678023' 2026-02-24T17:20:58.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:20:58.840 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803789 2026-02-24T17:20:58.840 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:20:58.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:20:58.842 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803789 2026-02-24T17:20:58.842 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:20:58.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803789 2026-02-24T17:20:58.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-24T17:20:58.843 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803789 2026-02-24T17:20:58.843 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:20:59.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-24T17:20:59.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:21:00.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:21:00.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:21:00.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803789 -lt 25769803789 2026-02-24T17:21:00.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:00.281 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607564 2026-02-24T17:21:00.281 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:00.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:21:00.283 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607564 2026-02-24T17:21:00.283 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:00.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607564 2026-02-24T17:21:00.285 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607564 2026-02-24T17:21:00.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607564' 2026-02-24T17:21:00.286 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:21:00.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607564 -lt 51539607564 2026-02-24T17:21:00.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:00.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411340 2026-02-24T17:21:00.503 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:00.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:21:00.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411340 2026-02-24T17:21:00.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:00.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411340 2026-02-24T17:21:00.506 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411340 2026-02-24T17:21:00.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411340' 2026-02-24T17:21:00.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:21:00.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411341 -lt 77309411340 2026-02-24T17:21:00.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:00.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117003 2026-02-24T17:21:00.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:00.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:21:00.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117003 2026-02-24T17:21:00.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:00.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117003 2026-02-24T17:21:00.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117003' 2026-02-24T17:21:00.727 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 115964117003 2026-02-24T17:21:00.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:21:00.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117003 -lt 115964117003 2026-02-24T17:21:00.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:00.939 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822667 2026-02-24T17:21:00.939 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:00.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:21:00.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822667 2026-02-24T17:21:00.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:00.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822667 2026-02-24T17:21:00.942 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 154618822667 2026-02-24T17:21:00.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822667' 2026-02-24T17:21:00.943 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:21:01.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822667 -lt 154618822667 2026-02-24T17:21:01.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:01.161 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-197568495625 2026-02-24T17:21:01.161 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:01.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:21:01.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-197568495625 2026-02-24T17:21:01.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:01.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=197568495625 2026-02-24T17:21:01.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 197568495625' 2026-02-24T17:21:01.165 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 197568495625 2026-02-24T17:21:01.165 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:21:01.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 197568495626 -lt 197568495625 2026-02-24T17:21:01.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:01.387 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-231928233992 2026-02-24T17:21:01.387 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:01.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:21:01.389 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-231928233992 2026-02-24T17:21:01.389 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:01.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=231928233992 2026-02-24T17:21:01.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 231928233992' 2026-02-24T17:21:01.390 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 231928233992 2026-02-24T17:21:01.391 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:21:01.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 231928233993 -lt 231928233992 2026-02-24T17:21:01.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:01.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 8-304942678023 2026-02-24T17:21:01.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:01.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=8 2026-02-24T17:21:01.604 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 8-304942678023 2026-02-24T17:21:01.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:01.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=304942678023 2026-02-24T17:21:01.606 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.8 seq 304942678023 2026-02-24T17:21:01.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.8 seq 304942678023' 2026-02-24T17:21:01.606 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-24T17:21:01.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 304942678023 -lt 304942678023 2026-02-24T17:21:01.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:21:01.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:21:01.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:21:02.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:21:02.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:21:02.120 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:21:02.120 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:21:02.120 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:21:02.120 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:21:02.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:21:02.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:21:02.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:21:02.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:21:02.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:21:02.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:21:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-24T17:21:02.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:21:02.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:21:02.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:21:02.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:21:02.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:21:02.895 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:21:02.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:21:03.201 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:21:03.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:21:03.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:21:03.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:21:03.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:21:03.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:21:03.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:21:03.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:21:03.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:21:03.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:21:03.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:21:03.527 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:21:03.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:21:03.529 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:21:03.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:21:03.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:21:03.532 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:21:03.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:03.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:21:03.533 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:21:03.533 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:03.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:21:03.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:21:03.535 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:21:03.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:21:03.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:03.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:03.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:21:03.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:21:03.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:21:03.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:21:03.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:21:03.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:21:03.562 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:21:03.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:21:03.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:21:03.566 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:21:03.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:21:03.567 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:21:03.568 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:21:03.568 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:21:03.568 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:21:03.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:03.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:21:03.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:21:03.570 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:03.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:21:03.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:21:03.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:21:03.573 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:21:03.573 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:03.573 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:03.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:21:03.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:21:03.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:21:03.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:21:03.576 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:21:03.576 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:03.576 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:03.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:21:03.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:03.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:21:03.629 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:21:03.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:21:03.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:21:03.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:21:03.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:21:03.630 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:21:03.631 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:21:03.631 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:21:03.631 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:21:03.631 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:21:03.631 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:03.631 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:03.631 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:21:03.632 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:21:03.632 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:21:03.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:21:03.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:21:03.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:21:03.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:21:03.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:21:03.690 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:21:03.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:21:03.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:21:03.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:21:03.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:21:03.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:21:03.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:21:03.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:21:03.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:21:03.859 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:21:03.860 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:03.860 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:03.860 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:03.860 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:03.860 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:03.860 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:03.861 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:21:03.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:21:03.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:21:03.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:21:03.986 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:21:03.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:21:04.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:21:04.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:21:04.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:21:04.997 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:21:04.998 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:04.998 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:04.998 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:21:04.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:21:04.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:21:05.047 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:21:05.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:21:05.055 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:21:03.627+0000 7f7875300ec0 0 load: jerasure load: lrc 2026-02-24T17:21:05.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-24T17:21:05.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-24T17:21:05.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-24T17:21:05.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:21:05.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:05.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-24T17:21:05.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-24T17:21:05.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:21:05.057 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-24T17:21:05.057 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-24T17:21:05.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:05.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:21:05.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:05.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:05.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:21:05.063 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:05.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c620c585-20be-4988-ad8b-61adf116025f 2026-02-24T17:21:05.064 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 c620c585-20be-4988-ad8b-61adf116025f 2026-02-24T17:21:05.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c620c585-20be-4988-ad8b-61adf116025f' 2026-02-24T17:21:05.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:05.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAB3p1pqWkBBRAAxv+qVZydvQayo4fMLwwP0w== 2026-02-24T17:21:05.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAB3p1pqWkBBRAAxv+qVZydvQayo4fMLwwP0w=="}' 2026-02-24T17:21:05.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c620c585-20be-4988-ad8b-61adf116025f -i td/test-erasure-eio/0/new.json 2026-02-24T17:21:05.190 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:05.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:21:05.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAB3p1pqWkBBRAAxv+qVZydvQayo4fMLwwP0w== --osd-uuid c620c585-20be-4988-ad8b-61adf116025f 2026-02-24T17:21:05.217 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:05.223+0000 7f25f1227a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:05.219 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:05.227+0000 7f25f1227a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:05.220 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:05.227+0000 7f25f1227a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:05.221 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:05.227+0000 7f25f1227a40 -1 bdev(0x56323a04e400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:05.221 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:05.227+0000 7f25f1227a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:21:05.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:21:05.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:05.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:21:05.947 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:21:05.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:06.101 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:21:06.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:21:06.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:06.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:06.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:06.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:06.118 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:06.127+0000 7fe7ef111a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:06.120 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:06.127+0000 7fe7ef111a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:06.121 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:06.127+0000 7fe7ef111a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:06.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:06.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:21:06.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:06.588 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:06.595+0000 7fe7ef111a40 -1 Falling back to public interface 2026-02-24T17:21:07.200 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:07.207+0000 7fe7ef111a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:21:07.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:07.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:07.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:07.437 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:07.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:07.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:21:07.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:08.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:08.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:08.660 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:08.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:08.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:08.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:21:08.873 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4078703736,v1:127.0.0.1:6803/4078703736] [v2:127.0.0.1:6804/4078703736,v1:127.0.0.1:6805/4078703736] exists,up c620c585-20be-4988-ad8b-61adf116025f 2026-02-24T17:21:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:08.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:08.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:21:08.880 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:08.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a0d8bfbb-6db7-4bf8-bfcf-38a8f781ea25 2026-02-24T17:21:08.881 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 a0d8bfbb-6db7-4bf8-bfcf-38a8f781ea25 2026-02-24T17:21:08.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a0d8bfbb-6db7-4bf8-bfcf-38a8f781ea25' 2026-02-24T17:21:08.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:08.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAE3p1poR2vNRAAGGzzhKF9uX3o7L4NM3zg3w== 2026-02-24T17:21:08.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAE3p1poR2vNRAAGGzzhKF9uX3o7L4NM3zg3w=="}' 2026-02-24T17:21:08.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a0d8bfbb-6db7-4bf8-bfcf-38a8f781ea25 -i td/test-erasure-eio/1/new.json 2026-02-24T17:21:09.109 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:09.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:21:09.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAE3p1poR2vNRAAGGzzhKF9uX3o7L4NM3zg3w== --osd-uuid a0d8bfbb-6db7-4bf8-bfcf-38a8f781ea25 2026-02-24T17:21:09.137 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:09.143+0000 7fd591c86a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:09.138 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:09.147+0000 7fd591c86a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:09.140 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:09.147+0000 7fd591c86a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:09.140 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:09.147+0000 7fd591c86a40 -1 bdev(0x55ebbee02400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:09.140 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:09.147+0000 7fd591c86a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:21:09.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:21:09.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:09.805 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:21:09.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:21:09.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:10.098 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:21:10.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:21:10.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:10.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:10.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:10.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:10.125 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:10.131+0000 7f87fb891a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:10.127 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:10.135+0000 7f87fb891a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:10.128 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:10.135+0000 7f87fb891a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:10.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:21:10.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:10.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:21:10.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:10.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:10.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:10.320 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:10.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:10.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:10.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:21:10.527 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:10.535+0000 7f87fb891a40 -1 Falling back to public interface 2026-02-24T17:21:10.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:11.176 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:11.183+0000 7f87fb891a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:21:11.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:11.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:11.538 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:11.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:11.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:11.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:21:11.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:12.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:12.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:12.756 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:12.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:12.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:12.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:21:12.977 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1730184839,v1:127.0.0.1:6811/1730184839] [v2:127.0.0.1:6812/1730184839,v1:127.0.0.1:6813/1730184839] exists,up a0d8bfbb-6db7-4bf8-bfcf-38a8f781ea25 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:12.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:12.981 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:12.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:12.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:12.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:12.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:12.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:12.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:12.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:21:12.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:12.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c65e1082-68e1-4617-9931-29edf53d48ee 2026-02-24T17:21:12.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c65e1082-68e1-4617-9931-29edf53d48ee' 2026-02-24T17:21:12.984 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 c65e1082-68e1-4617-9931-29edf53d48ee 2026-02-24T17:21:12.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:12.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAJ3p1pSw5NABAA5PLZNZaNDzFyoTl2JWZNQw== 2026-02-24T17:21:12.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAJ3p1pSw5NABAA5PLZNZaNDzFyoTl2JWZNQw=="}' 2026-02-24T17:21:12.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c65e1082-68e1-4617-9931-29edf53d48ee -i td/test-erasure-eio/2/new.json 2026-02-24T17:21:13.218 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:13.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:21:13.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAJ3p1pSw5NABAA5PLZNZaNDzFyoTl2JWZNQw== --osd-uuid c65e1082-68e1-4617-9931-29edf53d48ee 2026-02-24T17:21:13.255 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:13.263+0000 7f082501aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:13.256 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:13.263+0000 7f082501aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:13.257 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:13.263+0000 7f082501aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:13.258 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:13.263+0000 7f082501aa40 -1 bdev(0x558dd64a4400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:13.258 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:13.263+0000 7f082501aa40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:21:13.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:21:13.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:13.899 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:21:13.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:21:13.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:14.193 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:21:14.194 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:21:14.194 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:14.194 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:14.194 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:14.194 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:14.220 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:14.227+0000 7fcb5ea47a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:14.222 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:14.227+0000 7fcb5ea47a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:14.223 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:14.231+0000 7fcb5ea47a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:14.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:21:14.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:14.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:21:14.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:14.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:14.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:14.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:14.406 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:14.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:14.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:21:14.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:14.631 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:14.639+0000 7fcb5ea47a40 -1 Falling back to public interface 2026-02-24T17:21:15.311 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:15.319+0000 7fcb5ea47a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:21:15.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:15.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:15.626 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:15.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:15.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:15.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:21:15.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:16.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:16.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:16.841 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:16.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:16.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:16.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:21:17.057 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1017076467,v1:127.0.0.1:6819/1017076467] [v2:127.0.0.1:6820/1017076467,v1:127.0.0.1:6821/1017076467] exists,up c65e1082-68e1-4617-9931-29edf53d48ee 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:17.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:17.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:17.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:21:17.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:17.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=045701e3-d8cc-4867-b58b-8fe19b243ec6 2026-02-24T17:21:17.065 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 045701e3-d8cc-4867-b58b-8fe19b243ec6 2026-02-24T17:21:17.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 045701e3-d8cc-4867-b58b-8fe19b243ec6' 2026-02-24T17:21:17.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:17.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAN3p1pQw4HBRAAknDB458qMo6OHnC7cNCmow== 2026-02-24T17:21:17.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAN3p1pQw4HBRAAknDB458qMo6OHnC7cNCmow=="}' 2026-02-24T17:21:17.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 045701e3-d8cc-4867-b58b-8fe19b243ec6 -i td/test-erasure-eio/3/new.json 2026-02-24T17:21:17.292 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:21:17.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:21:17.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAN3p1pQw4HBRAAknDB458qMo6OHnC7cNCmow== --osd-uuid 045701e3-d8cc-4867-b58b-8fe19b243ec6 2026-02-24T17:21:17.321 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:17.327+0000 7f9b8b7b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:17.322 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:17.331+0000 7f9b8b7b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:17.323 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:17.331+0000 7f9b8b7b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:17.324 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:17.331+0000 7f9b8b7b0a40 -1 bdev(0x562b76c44400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:17.324 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:17.331+0000 7f9b8b7b0a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:21:17.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:21:17.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:17.970 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:21:17.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:21:17.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:18.272 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:21:18.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:21:18.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:18.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:18.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:18.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:18.291 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:18.299+0000 7f5cd3e45a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:18.292 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:18.299+0000 7f5cd3e45a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:18.293 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:18.299+0000 7f5cd3e45a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:18.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:21:18.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:18.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:21:18.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:18.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:18.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:18.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:18.482 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:18.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:18.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:21:18.696 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:18.703+0000 7f5cd3e45a40 -1 Falling back to public interface 2026-02-24T17:21:18.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:19.260 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:19.267+0000 7f5cd3e45a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:21:19.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:19.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:19.698 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:19.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:19.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:19.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:21:19.920 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/607668290,v1:127.0.0.1:6827/607668290] [v2:127.0.0.1:6828/607668290,v1:127.0.0.1:6829/607668290] exists,up 045701e3-d8cc-4867-b58b-8fe19b243ec6 2026-02-24T17:21:19.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:19.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:19.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:19.925 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:19.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-24T17:21:19.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:19.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=608df41b-354b-46c6-b466-da0f0ad6f618 2026-02-24T17:21:19.927 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 608df41b-354b-46c6-b466-da0f0ad6f618 2026-02-24T17:21:19.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 608df41b-354b-46c6-b466-da0f0ad6f618' 2026-02-24T17:21:19.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:19.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAP3p1pJv+NOBAAOGNXdj5dTy61G3Gv4jlFCA== 2026-02-24T17:21:19.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAP3p1pJv+NOBAAOGNXdj5dTy61G3Gv4jlFCA=="}' 2026-02-24T17:21:19.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 608df41b-354b-46c6-b466-da0f0ad6f618 -i td/test-erasure-eio/4/new.json 2026-02-24T17:21:20.156 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:21:20.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-24T17:21:20.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAP3p1pJv+NOBAAOGNXdj5dTy61G3Gv4jlFCA== --osd-uuid 608df41b-354b-46c6-b466-da0f0ad6f618 2026-02-24T17:21:20.186 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:20.191+0000 7ff948f07a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:20.187 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:20.195+0000 7ff948f07a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:20.188 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:20.195+0000 7ff948f07a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:20.188 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:20.195+0000 7ff948f07a40 -1 bdev(0x5599569ac400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:20.189 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:20.195+0000 7ff948f07a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-24T17:21:20.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-24T17:21:20.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:20.823 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:21:20.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:21:20.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:21.118 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:21:21.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:21:21.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:21.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:21.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:21.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:21.137 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:21.143+0000 7fbd02f90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:21.138 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:21.147+0000 7fbd02f90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:21.139 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:21.147+0000 7fbd02f90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:21.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:21:21.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:21.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:21:21.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:21.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:21.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:21.337 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:21.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:21.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:21.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:21:21.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:21.584 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:21.591+0000 7fbd02f90a40 -1 Falling back to public interface 2026-02-24T17:21:22.247 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:22.255+0000 7fbd02f90a40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:21:22.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:22.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:22.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:22.554 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:22.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:22.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:21:22.785 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:23.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:23.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:23.787 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:23.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:23.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:23.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:21:24.005 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 33 up_thru 34 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/4227909667,v1:127.0.0.1:6835/4227909667] [v2:127.0.0.1:6836/4227909667,v1:127.0.0.1:6837/4227909667] exists,up 608df41b-354b-46c6-b466-da0f0ad6f618 2026-02-24T17:21:24.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:24.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:24.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:24.007 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:24.008 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:24.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:24.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:24.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:24.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:24.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:24.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-24T17:21:24.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:24.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=066ab9aa-3af5-4f1d-a98a-19bff104d8a6 2026-02-24T17:21:24.011 INFO:tasks.workunit.client.0.trial066.stdout:add osd5 066ab9aa-3af5-4f1d-a98a-19bff104d8a6 2026-02-24T17:21:24.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 066ab9aa-3af5-4f1d-a98a-19bff104d8a6' 2026-02-24T17:21:24.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:24.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAU3p1pRv/wARAAM0Cslo5LCwCCbAeYcFBFeg== 2026-02-24T17:21:24.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAU3p1pRv/wARAAM0Cslo5LCwCCbAeYcFBFeg=="}' 2026-02-24T17:21:24.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 066ab9aa-3af5-4f1d-a98a-19bff104d8a6 -i td/test-erasure-eio/5/new.json 2026-02-24T17:21:24.244 INFO:tasks.workunit.client.0.trial066.stdout:5 2026-02-24T17:21:24.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-24T17:21:24.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAU3p1pRv/wARAAM0Cslo5LCwCCbAeYcFBFeg== --osd-uuid 066ab9aa-3af5-4f1d-a98a-19bff104d8a6 2026-02-24T17:21:24.273 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:24.278+0000 7fc7cababa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:24.274 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:24.282+0000 7fc7cababa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:24.276 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:24.282+0000 7fc7cababa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:24.276 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:24.282+0000 7fc7cababa40 -1 bdev(0x55b16015a400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:24.276 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:24.282+0000 7fc7cababa40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-24T17:21:24.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-24T17:21:24.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:24.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-24T17:21:24.942 INFO:tasks.workunit.client.0.trial066.stdout:adding osd5 key to auth repository 2026-02-24T17:21:24.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:25.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-24T17:21:25.245 INFO:tasks.workunit.client.0.trial066.stdout:start osd.5 2026-02-24T17:21:25.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:25.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:25.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:25.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:25.272 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:25.278+0000 7faf0f592a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:25.274 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:25.278+0000 7faf0f592a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:25.275 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:25.282+0000 7faf0f592a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:25.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:21:25.663 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:25.670+0000 7faf0f592a40 -1 Falling back to public interface 2026-02-24T17:21:25.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:26.383 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:26.390+0000 7faf0f592a40 -1 osd.5 0 log_to_monitors true 2026-02-24T17:21:26.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:26.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:26.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:26.674 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:26.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:26.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:21:26.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:27.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:27.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:27.901 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:27.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:27.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:27.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stdout:osd.5 up in weight 1 up_from 41 up_thru 42 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/805763196,v1:127.0.0.1:6843/805763196] [v2:127.0.0.1:6844/805763196,v1:127.0.0.1:6845/805763196] exists,up 066ab9aa-3af5-4f1d-a98a-19bff104d8a6 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:28.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:28.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-24T17:21:28.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:28.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4e779381-21e9-4121-a6d5-8e7fe399f76c 2026-02-24T17:21:28.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 4e779381-21e9-4121-a6d5-8e7fe399f76c' 2026-02-24T17:21:28.125 INFO:tasks.workunit.client.0.trial066.stdout:add osd6 4e779381-21e9-4121-a6d5-8e7fe399f76c 2026-02-24T17:21:28.125 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:28.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAY3p1pw+3UCBAAnSgltwKWLTTiCyyXPCCwsg== 2026-02-24T17:21:28.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAY3p1pw+3UCBAAnSgltwKWLTTiCyyXPCCwsg=="}' 2026-02-24T17:21:28.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4e779381-21e9-4121-a6d5-8e7fe399f76c -i td/test-erasure-eio/6/new.json 2026-02-24T17:21:28.358 INFO:tasks.workunit.client.0.trial066.stdout:6 2026-02-24T17:21:28.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-24T17:21:28.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAY3p1pw+3UCBAAnSgltwKWLTTiCyyXPCCwsg== --osd-uuid 4e779381-21e9-4121-a6d5-8e7fe399f76c 2026-02-24T17:21:28.387 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:28.394+0000 7fdb1c002a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:28.389 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:28.394+0000 7fdb1c002a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:28.390 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:28.394+0000 7fdb1c002a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:28.390 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:28.394+0000 7fdb1c002a40 -1 bdev(0x5630f9cc8400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:28.390 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:28.394+0000 7fdb1c002a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-24T17:21:29.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-24T17:21:29.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:29.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-24T17:21:29.071 INFO:tasks.workunit.client.0.trial066.stdout:adding osd6 key to auth repository 2026-02-24T17:21:29.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:29.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-24T17:21:29.371 INFO:tasks.workunit.client.0.trial066.stdout:start osd.6 2026-02-24T17:21:29.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:29.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:29.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:29.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:29.397 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:29.402+0000 7fd62b21aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:29.398 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:29.406+0000 7fd62b21aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:29.400 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:29.406+0000 7fd62b21aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:29.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-24T17:21:29.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:29.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-24T17:21:29.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:29.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:29.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:29.590 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:29.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:29.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:29.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:21:29.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:29.852 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:29.858+0000 7fd62b21aa40 -1 Falling back to public interface 2026-02-24T17:21:30.480 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:30.486+0000 7fd62b21aa40 -1 osd.6 0 log_to_monitors true 2026-02-24T17:21:30.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:30.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:30.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:30.803 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:30.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:30.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:21:31.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:32.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:32.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:32.044 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:32.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:32.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:32.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:21:32.258 INFO:tasks.workunit.client.0.trial066.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3780839009,v1:127.0.0.1:6851/3780839009] [v2:127.0.0.1:6852/3780839009,v1:127.0.0.1:6853/3780839009] exists,up 4e779381-21e9-4121-a6d5-8e7fe399f76c 2026-02-24T17:21:32.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:32.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:32.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:32.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:21:32.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:21:32.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:21:32.262 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:21:32.262 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:32.262 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:32.262 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:21:32.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:21:32.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:21:32.317 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:21:32.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:21:32.325 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:21:06.599+0000 7fe7ef111a40 0 load: jerasure load: lrc 2026-02-24T17:21:32.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:21:32.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:21:32.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-24T17:21:32.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:21:32.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:21:32.951 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:21:32.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:21:33.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:21:33.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:21:33.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:21:33.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:21:33.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:21:33.965 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:21:33.965 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:21:33.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:21:33.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:21:33.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:21:34.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:21:34.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:21:34.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:21:34.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:21:34.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:21:34.074 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:21:34.284 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:21:34.284 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:34.285 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:21:34.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803784 2026-02-24T17:21:34.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803784 2026-02-24T17:21:34.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784' 2026-02-24T17:21:34.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:34.364 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:21:34.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607559 2026-02-24T17:21:34.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607559 2026-02-24T17:21:34.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607559' 2026-02-24T17:21:34.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:34.443 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:21:34.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411334 2026-02-24T17:21:34.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411334 2026-02-24T17:21:34.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411334' 2026-02-24T17:21:34.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:34.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:21:34.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182405 2026-02-24T17:21:34.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182405 2026-02-24T17:21:34.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411334 3-107374182405' 2026-02-24T17:21:34.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:34.604 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:21:34.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920774 2026-02-24T17:21:34.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920774 2026-02-24T17:21:34.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411334 3-107374182405 4-141733920774' 2026-02-24T17:21:34.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:34.682 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:21:34.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=176093659140 2026-02-24T17:21:34.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 176093659140 2026-02-24T17:21:34.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411334 3-107374182405 4-141733920774 5-176093659140' 2026-02-24T17:21:34.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:34.761 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:21:34.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397507 2026-02-24T17:21:34.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397507 2026-02-24T17:21:34.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607559 2-77309411334 3-107374182405 4-141733920774 5-176093659140 6-210453397507' 2026-02-24T17:21:34.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:34.840 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803784 2026-02-24T17:21:34.840 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:34.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:21:34.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803784 2026-02-24T17:21:34.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:34.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803784 2026-02-24T17:21:34.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-24T17:21:34.843 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803784 2026-02-24T17:21:34.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:21:35.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-24T17:21:35.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:21:36.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:21:36.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:21:36.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-24T17:21:36.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:36.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607559 2026-02-24T17:21:36.274 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:36.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:21:36.275 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607559 2026-02-24T17:21:36.275 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:36.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607559 2026-02-24T17:21:36.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-24T17:21:36.277 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607559 2026-02-24T17:21:36.277 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:21:36.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-24T17:21:36.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:36.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411334 2026-02-24T17:21:36.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:36.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:21:36.499 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411334 2026-02-24T17:21:36.499 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:36.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411334 2026-02-24T17:21:36.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-24T17:21:36.501 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411334 2026-02-24T17:21:36.501 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:21:36.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-24T17:21:36.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:36.717 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182405 2026-02-24T17:21:36.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:36.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:21:36.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182405 2026-02-24T17:21:36.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:36.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182405 2026-02-24T17:21:36.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182405' 2026-02-24T17:21:36.721 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182405 2026-02-24T17:21:36.721 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:21:36.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182406 -lt 107374182405 2026-02-24T17:21:36.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:36.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920774 2026-02-24T17:21:36.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:36.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:21:36.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920774 2026-02-24T17:21:36.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:36.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920774 2026-02-24T17:21:36.944 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 141733920774 2026-02-24T17:21:36.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920774' 2026-02-24T17:21:36.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:21:37.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-24T17:21:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:37.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-176093659140 2026-02-24T17:21:37.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:37.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:21:37.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-176093659140 2026-02-24T17:21:37.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:37.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=176093659140 2026-02-24T17:21:37.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 176093659140' 2026-02-24T17:21:37.174 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 176093659140 2026-02-24T17:21:37.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:21:37.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 176093659140 -lt 176093659140 2026-02-24T17:21:37.393 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:37.393 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-210453397507 2026-02-24T17:21:37.393 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:37.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:21:37.395 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-210453397507 2026-02-24T17:21:37.395 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397507 2026-02-24T17:21:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-24T17:21:37.397 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 210453397507 2026-02-24T17:21:37.397 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:21:37.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-24T17:21:37.622 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:21:37.622 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:21:37.622 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:21:37.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:21:37.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:21:37.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:21:37.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:21:37.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:21:37.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:21:37.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:21:37.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:21:38.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:21:38.108 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:21:38.108 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:21:38.109 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:21:38.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-24T17:21:38.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-24T17:21:38.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:21:38.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:21:38.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-24T17:21:38.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:21:38.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:21:38.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-24T17:21:38.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-24T17:21:38.723 INFO:tasks.workunit.client.0.trial066.stderr:osd.0 is already out. 2026-02-24T17:21:38.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-24T17:21:38.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:21:38.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:21:38.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:21:38.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:21:38.735 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:21:38.735 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:21:38.735 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:21:38.735 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:21:38.735 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:21:38.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:21:38.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:21:38.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:21:38.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:21:38.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:21:38.845 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:21:39.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:21:39.052 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:21:39.052 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:21:39.053 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:21:39.053 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:21:39.053 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:21:39.053 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:21:39.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:21:39.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:39.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:21:39.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803787 2026-02-24T17:21:39.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803787 2026-02-24T17:21:39.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787' 2026-02-24T17:21:39.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:39.134 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:21:39.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607562 2026-02-24T17:21:39.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607562 2026-02-24T17:21:39.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562' 2026-02-24T17:21:39.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:39.212 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:21:39.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411337 2026-02-24T17:21:39.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411337 2026-02-24T17:21:39.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337' 2026-02-24T17:21:39.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:39.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:21:39.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182408 2026-02-24T17:21:39.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182408 2026-02-24T17:21:39.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-107374182408' 2026-02-24T17:21:39.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:39.375 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:21:39.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=141733920777 2026-02-24T17:21:39.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 141733920777 2026-02-24T17:21:39.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-107374182408 4-141733920777' 2026-02-24T17:21:39.457 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:39.457 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:21:39.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=176093659143 2026-02-24T17:21:39.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 176093659143 2026-02-24T17:21:39.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-107374182408 4-141733920777 5-176093659143' 2026-02-24T17:21:39.534 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:21:39.534 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:21:39.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397510 2026-02-24T17:21:39.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397510 2026-02-24T17:21:39.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411337 3-107374182408 4-141733920777 5-176093659143 6-210453397510' 2026-02-24T17:21:39.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:39.613 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803787 2026-02-24T17:21:39.613 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:39.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:21:39.614 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803787 2026-02-24T17:21:39.614 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:39.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803787 2026-02-24T17:21:39.616 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803787 2026-02-24T17:21:39.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-24T17:21:39.616 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:21:39.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803787 2026-02-24T17:21:39.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:21:40.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:21:40.836 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:21:41.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-24T17:21:41.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:41.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607562 2026-02-24T17:21:41.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:41.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:21:41.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607562 2026-02-24T17:21:41.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:41.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607562 2026-02-24T17:21:41.060 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607562 2026-02-24T17:21:41.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607562' 2026-02-24T17:21:41.060 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:21:41.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607562 -lt 51539607562 2026-02-24T17:21:41.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:41.284 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411337 2026-02-24T17:21:41.284 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:41.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:21:41.286 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411337 2026-02-24T17:21:41.286 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:41.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411337 2026-02-24T17:21:41.287 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411337 2026-02-24T17:21:41.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-24T17:21:41.288 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:21:41.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-24T17:21:41.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:41.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182408 2026-02-24T17:21:41.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:41.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:21:41.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182408 2026-02-24T17:21:41.506 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:41.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182408 2026-02-24T17:21:41.507 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182408 2026-02-24T17:21:41.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182408' 2026-02-24T17:21:41.508 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:21:41.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182408 -lt 107374182408 2026-02-24T17:21:41.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:41.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-141733920777 2026-02-24T17:21:41.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:41.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:21:41.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-141733920777 2026-02-24T17:21:41.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:41.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=141733920777 2026-02-24T17:21:41.729 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 141733920777 2026-02-24T17:21:41.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 141733920777' 2026-02-24T17:21:41.729 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:21:41.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 141733920777 -lt 141733920777 2026-02-24T17:21:41.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:41.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-176093659143 2026-02-24T17:21:41.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:41.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:21:41.946 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-176093659143 2026-02-24T17:21:41.947 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:41.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=176093659143 2026-02-24T17:21:41.948 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 176093659143 2026-02-24T17:21:41.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 176093659143' 2026-02-24T17:21:41.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:21:42.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 176093659143 -lt 176093659143 2026-02-24T17:21:42.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:21:42.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-210453397510 2026-02-24T17:21:42.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:21:42.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:21:42.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-210453397510 2026-02-24T17:21:42.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:21:42.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397510 2026-02-24T17:21:42.179 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 210453397510 2026-02-24T17:21:42.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 210453397510' 2026-02-24T17:21:42.180 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:21:42.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397511 -lt 210453397510 2026-02-24T17:21:42.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:21:42.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:21:42.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:21:42.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:21:42.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:21:42.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:21:42.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:21:42.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:21:42.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:21:42.698 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:21:42.698 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:21:42.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:21:42.910 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:21:42.911 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:21:42.911 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:21:43.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:21:43.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:21:43.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:21:43.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-24T17:21:43.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:21:43.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:21:43.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-24T17:21:43.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:21:43.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:21:43.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-24T17:21:43.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:21:43.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:21:43.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-24T17:21:43.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:21:43.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:21:43.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-24T17:21:43.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:21:43.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:21:43.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-24T17:21:43.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:21:43.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:21:43.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-24T17:21:43.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:21:43.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:21:43.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-24T17:21:43.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:21:43.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:21:43.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-24T17:21:43.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:21:43.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:21:43.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:21:43.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:21:43.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:21:43.527 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:21:43.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:21:43.814 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:21:43.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:21:43.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:21:43.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:21:43.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:21:43.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:21:43.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:21:43.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:21:43.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:21:43.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:21:44.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:21:44.138 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:21:44.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:21:44.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:21:44.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:21:44.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:21:44.142 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:21:44.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:44.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:21:44.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:21:44.144 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:44.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:21:44.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:21:44.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:21:44.166 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:21:44.166 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:44.166 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:44.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:21:44.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:21:44.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:21:44.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:21:44.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:21:44.171 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:21:44.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:21:44.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:21:44.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:21:44.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:21:44.173 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:21:44.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:44.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:21:44.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:21:44.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:21:44.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:21:44.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:21:44.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:21:44.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:21:44.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:44.179 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:44.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:21:44.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:21:44.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:21:44.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:21:44.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:21:44.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:44.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:44.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:21:44.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:21:44.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:21:44.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:21:44.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:21:44.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:21:44.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:44.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:44.206 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:44.206 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:44.206 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:44.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:44.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:21:44.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:21:44.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:21:44.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:21:44.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:21:44.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:44.236 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:21:44.237 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:21:44.237 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:21:44.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:21:44.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:21:44.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:21:44.293 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:21:44.293 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:21:44.293 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:21:44.293 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:21:44.293 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:21:44.293 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:21:44.294 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:44.294 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:44.294 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:21:44.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:21:44.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:21:44.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:21:44.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:21:44.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:21:44.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:21:44.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:21:44.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:21:44.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:21:44.465 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:21:44.465 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:44.465 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:44.465 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:44.465 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:44.465 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:44.465 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:44.466 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:21:44.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:21:44.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:21:44.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:21:44.591 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:21:44.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:21:45.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:21:45.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:21:45.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:21:45.602 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:21:45.602 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:45.602 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:45.603 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:21:45.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:21:45.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:21:45.652 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:21:45.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:21:45.660 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:21:44.230+0000 7f9d9efdfec0 0 load: jerasure load: lrc 2026-02-24T17:21:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-24T17:21:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-24T17:21:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-24T17:21:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:21:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-24T17:21:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-24T17:21:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:21:45.663 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-24T17:21:45.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-24T17:21:45.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:45.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:21:45.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:45.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:45.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:21:45.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:45.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:21:45.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:45.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:45.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:45.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:45.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:21:45.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:21:45.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:45.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:45.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:45.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:45.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:45.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:45.674 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:45.674 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:45.674 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:45.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:45.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:45.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:45.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:45.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:45.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:45.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:21:45.676 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:45.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=36981d44-0031-4f22-a182-417691a98cab 2026-02-24T17:21:45.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 36981d44-0031-4f22-a182-417691a98cab' 2026-02-24T17:21:45.677 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 36981d44-0031-4f22-a182-417691a98cab 2026-02-24T17:21:45.677 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:45.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAp3p1pff0dKRAAsCDlBK0UOQH/xOhGU7hOjA== 2026-02-24T17:21:45.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAp3p1pff0dKRAAsCDlBK0UOQH/xOhGU7hOjA=="}' 2026-02-24T17:21:45.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 36981d44-0031-4f22-a182-417691a98cab -i td/test-erasure-eio/0/new.json 2026-02-24T17:21:45.793 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:45.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:21:45.803 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAp3p1pff0dKRAAsCDlBK0UOQH/xOhGU7hOjA== --osd-uuid 36981d44-0031-4f22-a182-417691a98cab 2026-02-24T17:21:45.821 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:45.826+0000 7fa1c3edfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:45.823 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:45.830+0000 7fa1c3edfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:45.824 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:45.830+0000 7fa1c3edfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:45.824 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:45.830+0000 7fa1c3edfa40 -1 bdev(0x563269caa400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:45.825 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:45.830+0000 7fa1c3edfa40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:21:46.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:21:46.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:46.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:21:46.523 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:21:46.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:46.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:21:46.675 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:21:46.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:46.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:46.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:46.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:46.691 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:46.698+0000 7fcd33716a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:46.692 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:46.698+0000 7fcd33716a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:46.693 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:46.698+0000 7fcd33716a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:46.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:21:47.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:47.136 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:47.142+0000 7fcd33716a40 -1 Falling back to public interface 2026-02-24T17:21:47.791 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:47.798+0000 7fcd33716a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:21:48.003 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:48.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:48.004 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:48.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:48.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:48.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:21:48.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:49.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:49.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:49.227 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:49.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:49.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:49.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:21:49.449 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3154997709,v1:127.0.0.1:6803/3154997709] [v2:127.0.0.1:6804/3154997709,v1:127.0.0.1:6805/3154997709] exists,up 36981d44-0031-4f22-a182-417691a98cab 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:49.450 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:49.451 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:49.451 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:49.451 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:49.451 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:49.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:49.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:49.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:49.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:21:49.453 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:49.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=135e79fe-e444-4fd3-a469-0d37f22a6ec2 2026-02-24T17:21:49.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 135e79fe-e444-4fd3-a469-0d37f22a6ec2' 2026-02-24T17:21:49.454 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 135e79fe-e444-4fd3-a469-0d37f22a6ec2 2026-02-24T17:21:49.454 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:49.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAt3p1pXopnHBAADC0qy3Ez0C8u22un7hog2Q== 2026-02-24T17:21:49.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAt3p1pXopnHBAADC0qy3Ez0C8u22un7hog2Q=="}' 2026-02-24T17:21:49.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 135e79fe-e444-4fd3-a469-0d37f22a6ec2 -i td/test-erasure-eio/1/new.json 2026-02-24T17:21:49.681 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:49.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:21:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAt3p1pXopnHBAADC0qy3Ez0C8u22un7hog2Q== --osd-uuid 135e79fe-e444-4fd3-a469-0d37f22a6ec2 2026-02-24T17:21:49.709 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:49.714+0000 7f38720a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:49.711 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:49.718+0000 7f38720a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:49.712 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:49.718+0000 7f38720a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:49.712 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:49.718+0000 7f38720a3a40 -1 bdev(0x55c77bfd6400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:49.712 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:49.718+0000 7f38720a3a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:21:50.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:21:50.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:50.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:21:50.347 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:21:50.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:50.638 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:21:50.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:21:50.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:50.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:50.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:50.639 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:50.666 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:50.674+0000 7f3bdde4aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:50.668 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:50.674+0000 7f3bdde4aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:50.669 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:50.674+0000 7f3bdde4aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:50.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:21:50.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:50.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:21:50.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:50.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:50.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:50.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:50.862 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:50.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:50.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:21:51.076 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:51.124 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:51.130+0000 7f3bdde4aa40 -1 Falling back to public interface 2026-02-24T17:21:51.811 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:51.818+0000 7f3bdde4aa40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:21:52.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:52.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:52.078 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:52.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:52.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:52.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:21:52.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:53.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:53.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:53.300 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:53.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:53.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:53.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:21:53.512 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2017819781,v1:127.0.0.1:6811/2017819781] [v2:127.0.0.1:6812/2017819781,v1:127.0.0.1:6813/2017819781] exists,up 135e79fe-e444-4fd3-a469-0d37f22a6ec2 2026-02-24T17:21:53.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:53.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:53.517 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:53.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:53.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:53.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:21:53.519 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:53.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4aa24946-72c6-465f-9215-228d27ff4d6d 2026-02-24T17:21:53.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4aa24946-72c6-465f-9215-228d27ff4d6d' 2026-02-24T17:21:53.520 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 4aa24946-72c6-465f-9215-228d27ff4d6d 2026-02-24T17:21:53.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:53.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAx3p1pHNo2IBAA1QTozURZ/splwrX5aL4zKQ== 2026-02-24T17:21:53.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAx3p1pHNo2IBAA1QTozURZ/splwrX5aL4zKQ=="}' 2026-02-24T17:21:53.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4aa24946-72c6-465f-9215-228d27ff4d6d -i td/test-erasure-eio/2/new.json 2026-02-24T17:21:53.742 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:53.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:21:53.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAx3p1pHNo2IBAA1QTozURZ/splwrX5aL4zKQ== --osd-uuid 4aa24946-72c6-465f-9215-228d27ff4d6d 2026-02-24T17:21:53.770 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:53.774+0000 7fe317453a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:53.771 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:53.778+0000 7fe317453a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:53.772 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:53.778+0000 7fe317453a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:53.773 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:53.778+0000 7fe317453a40 -1 bdev(0x55976f492400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:53.773 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:53.778+0000 7fe317453a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:21:54.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:21:54.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:54.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:21:54.437 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:21:54.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:54.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:21:54.739 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:21:54.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:54.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:54.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:54.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:54.757 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:54.762+0000 7fee1386ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:54.759 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:54.766+0000 7fee1386ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:54.760 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:54.766+0000 7fee1386ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:54.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:21:54.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:54.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:21:54.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:54.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:54.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:54.963 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:54.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:54.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:54.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:21:55.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:55.212 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:55.218+0000 7fee1386ca40 -1 Falling back to public interface 2026-02-24T17:21:55.838 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:55.842+0000 7fee1386ca40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:21:56.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:56.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:56.176 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:21:56.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:21:56.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:56.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:21:56.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:57.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:21:57.398 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:21:57.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:57.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:21:57.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:57.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:21:57.618 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3748288153,v1:127.0.0.1:6819/3748288153] [v2:127.0.0.1:6820/3748288153,v1:127.0.0.1:6821/3748288153] exists,up 4aa24946-72c6-465f-9215-228d27ff4d6d 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:21:57.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:21:57.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:21:57.620 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:21:57.620 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:21:57.620 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:21:57.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:21:57.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:21:57.622 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:21:57.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bd7101e2-b80a-417b-b64f-b6688fa3ab80 2026-02-24T17:21:57.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 bd7101e2-b80a-417b-b64f-b6688fa3ab80' 2026-02-24T17:21:57.624 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 bd7101e2-b80a-417b-b64f-b6688fa3ab80 2026-02-24T17:21:57.624 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:21:57.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA13p1pli2QJhAAPo4ryqY36CucK9GH6cvJ/Q== 2026-02-24T17:21:57.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA13p1pli2QJhAAPo4ryqY36CucK9GH6cvJ/Q=="}' 2026-02-24T17:21:57.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bd7101e2-b80a-417b-b64f-b6688fa3ab80 -i td/test-erasure-eio/3/new.json 2026-02-24T17:21:57.868 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:21:57.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:21:57.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA13p1pli2QJhAAPo4ryqY36CucK9GH6cvJ/Q== --osd-uuid bd7101e2-b80a-417b-b64f-b6688fa3ab80 2026-02-24T17:21:57.906 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:57.914+0000 7fea6b0d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:57.908 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:57.914+0000 7fea6b0d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:57.909 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:57.914+0000 7fea6b0d0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:57.910 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:57.914+0000 7fea6b0d0a40 -1 bdev(0x55e1def2c400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:21:57.910 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:57.914+0000 7fea6b0d0a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:21:58.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:21:58.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:21:58.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:21:58.575 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:21:58.575 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:21:58.857 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:21:58.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:21:58.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:21:58.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:21:58.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:21:58.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:21:58.883 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:58.890+0000 7fd7e4f34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:58.885 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:58.890+0000 7fd7e4f34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:58.886 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:58.890+0000 7fd7e4f34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:21:59.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:21:59.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:21:59.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:21:59.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:21:59.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:21:59.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:21:59.081 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:21:59.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:21:59.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:21:59.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:21:59.296 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:59.302+0000 7fd7e4f34a40 -1 Falling back to public interface 2026-02-24T17:21:59.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:21:59.952 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:21:59.958+0000 7fd7e4f34a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:22:00.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:00.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:00.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:00.303 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:00.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:00.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:22:00.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:01.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:01.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:01.521 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:01.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:22:01.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:01.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:22:01.743 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2447998076,v1:127.0.0.1:6827/2447998076] [v2:127.0.0.1:6828/2447998076,v1:127.0.0.1:6829/2447998076] exists,up bd7101e2-b80a-417b-b64f-b6688fa3ab80 2026-02-24T17:22:01.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:01.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:01.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:01.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:01.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-24T17:22:01.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:01.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:01.749 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:01.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-24T17:22:01.751 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:01.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=811f7752-c43a-47af-9660-a2025fc49485 2026-02-24T17:22:01.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 811f7752-c43a-47af-9660-a2025fc49485' 2026-02-24T17:22:01.751 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 811f7752-c43a-47af-9660-a2025fc49485 2026-02-24T17:22:01.752 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:01.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA53p1pcRADLhAArD0wywaexb4YeMAL+pVtDQ== 2026-02-24T17:22:01.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA53p1pcRADLhAArD0wywaexb4YeMAL+pVtDQ=="}' 2026-02-24T17:22:01.763 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 811f7752-c43a-47af-9660-a2025fc49485 -i td/test-erasure-eio/4/new.json 2026-02-24T17:22:01.988 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:22:01.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-24T17:22:02.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA53p1pcRADLhAArD0wywaexb4YeMAL+pVtDQ== --osd-uuid 811f7752-c43a-47af-9660-a2025fc49485 2026-02-24T17:22:02.026 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.034+0000 7f30c2b5da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:02.028 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.034+0000 7f30c2b5da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:02.029 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.034+0000 7f30c2b5da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:02.029 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.034+0000 7f30c2b5da40 -1 bdev(0x55b98c470400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:02.030 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.034+0000 7f30c2b5da40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-24T17:22:02.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-24T17:22:02.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:02.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:22:02.657 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:22:02.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:02.964 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:22:02.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:22:02.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:02.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:02.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:02.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:02.981 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.986+0000 7f922f493a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:02.983 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.990+0000 7f922f493a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:02.984 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:02.990+0000 7f922f493a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:03.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:22:03.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:03.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:22:03.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:03.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:03.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:03.187 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:03.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:03.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:03.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:22:03.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:03.400 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:03.406+0000 7f922f493a40 -1 Falling back to public interface 2026-02-24T17:22:04.046 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:04.050+0000 7f922f493a40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:22:04.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:04.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:04.399 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:04.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:04.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:04.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:22:04.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:05.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:05.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:05.616 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:05.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:22:05.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:05.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:22:05.826 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 36 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2675642442,v1:127.0.0.1:6835/2675642442] [v2:127.0.0.1:6836/2675642442,v1:127.0.0.1:6837/2675642442] exists,up 811f7752-c43a-47af-9660-a2025fc49485 2026-02-24T17:22:05.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:05.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:05.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:05.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:05.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-24T17:22:05.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:05.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-24T17:22:05.833 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:05.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e984254a-dfd9-4d7a-a1ed-13069bd86ef0 2026-02-24T17:22:05.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 e984254a-dfd9-4d7a-a1ed-13069bd86ef0' 2026-02-24T17:22:05.834 INFO:tasks.workunit.client.0.trial066.stdout:add osd5 e984254a-dfd9-4d7a-a1ed-13069bd86ef0 2026-02-24T17:22:05.835 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:05.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA93p1p2sHvMhAAepC9ROz8x6KcX0HPOCSU+w== 2026-02-24T17:22:05.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA93p1p2sHvMhAAepC9ROz8x6KcX0HPOCSU+w=="}' 2026-02-24T17:22:05.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e984254a-dfd9-4d7a-a1ed-13069bd86ef0 -i td/test-erasure-eio/5/new.json 2026-02-24T17:22:06.058 INFO:tasks.workunit.client.0.trial066.stdout:5 2026-02-24T17:22:06.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-24T17:22:06.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA93p1p2sHvMhAAepC9ROz8x6KcX0HPOCSU+w== --osd-uuid e984254a-dfd9-4d7a-a1ed-13069bd86ef0 2026-02-24T17:22:06.086 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:06.090+0000 7f64f3520a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:06.088 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:06.094+0000 7f64f3520a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:06.089 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:06.094+0000 7f64f3520a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:06.090 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:06.094+0000 7f64f3520a40 -1 bdev(0x5571df456400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:06.090 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:06.094+0000 7f64f3520a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-24T17:22:06.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-24T17:22:06.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:06.774 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-24T17:22:06.774 INFO:tasks.workunit.client.0.trial066.stdout:adding osd5 key to auth repository 2026-02-24T17:22:06.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:07.062 INFO:tasks.workunit.client.0.trial066.stdout:start osd.5 2026-02-24T17:22:07.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-24T17:22:07.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:07.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:07.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:07.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:07.091 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:07.098+0000 7f36ed2c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:07.092 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:07.098+0000 7f36ed2c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:07.093 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:07.098+0000 7f36ed2c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:07.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-24T17:22:07.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:07.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-24T17:22:07.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:07.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:07.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:07.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:07.279 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:07.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:07.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:22:07.448 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:07.454+0000 7f36ed2c2a40 -1 Falling back to public interface 2026-02-24T17:22:07.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:08.074 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:08.078+0000 7f36ed2c2a40 -1 osd.5 0 log_to_monitors true 2026-02-24T17:22:08.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:08.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:08.492 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:08.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:08.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:08.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:22:08.718 INFO:tasks.workunit.client.0.trial066.stdout:osd.5 up in weight 1 up_from 44 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3161130549,v1:127.0.0.1:6843/3161130549] [v2:127.0.0.1:6844/3161130549,v1:127.0.0.1:6845/3161130549] exists,up e984254a-dfd9-4d7a-a1ed-13069bd86ef0 2026-02-24T17:22:08.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:08.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:08.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:08.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-24T17:22:08.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:08.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-24T17:22:08.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:08.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4afa7113-1de4-4b0a-bb60-b349f2fef801 2026-02-24T17:22:08.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 4afa7113-1de4-4b0a-bb60-b349f2fef801' 2026-02-24T17:22:08.725 INFO:tasks.workunit.client.0.trial066.stdout:add osd6 4afa7113-1de4-4b0a-bb60-b349f2fef801 2026-02-24T17:22:08.726 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:08.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBA3p1pmEtzLBAAu5cnBN3b9uEolNyHmw3gGw== 2026-02-24T17:22:08.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBA3p1pmEtzLBAAu5cnBN3b9uEolNyHmw3gGw=="}' 2026-02-24T17:22:08.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4afa7113-1de4-4b0a-bb60-b349f2fef801 -i td/test-erasure-eio/6/new.json 2026-02-24T17:22:08.963 INFO:tasks.workunit.client.0.trial066.stdout:6 2026-02-24T17:22:08.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-24T17:22:08.974 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBA3p1pmEtzLBAAu5cnBN3b9uEolNyHmw3gGw== --osd-uuid 4afa7113-1de4-4b0a-bb60-b349f2fef801 2026-02-24T17:22:08.993 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:08.998+0000 7fa0a7819a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:08.994 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:08.998+0000 7fa0a7819a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:08.995 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:09.002+0000 7fa0a7819a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:08.996 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:09.002+0000 7fa0a7819a40 -1 bdev(0x562c63b1a400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:08.996 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:09.002+0000 7fa0a7819a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-24T17:22:09.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-24T17:22:09.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:09.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-24T17:22:09.666 INFO:tasks.workunit.client.0.trial066.stdout:adding osd6 key to auth repository 2026-02-24T17:22:09.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:09.957 INFO:tasks.workunit.client.0.trial066.stdout:start osd.6 2026-02-24T17:22:09.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-24T17:22:09.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:09.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:09.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:09.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:09.984 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:09.990+0000 7f99ca4bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:09.986 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:09.990+0000 7f99ca4bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:09.987 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:09.994+0000 7f99ca4bea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:10.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-24T17:22:10.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:10.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-24T17:22:10.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:10.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:10.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:10.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:10.170 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:10.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:10.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:22:10.340 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:10.346+0000 7f99ca4bea40 -1 Falling back to public interface 2026-02-24T17:22:10.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:11.016 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:11.022+0000 7f99ca4bea40 -1 osd.6 0 log_to_monitors true 2026-02-24T17:22:11.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:11.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:11.384 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:11.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:11.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:11.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:22:11.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:12.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:12.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:12.606 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:12.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:22:12.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:12.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:22:12.824 INFO:tasks.workunit.client.0.trial066.stdout:osd.6 up in weight 1 up_from 51 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/3424611628,v1:127.0.0.1:6851/3424611628] [v2:127.0.0.1:6852/3424611628,v1:127.0.0.1:6853/3424611628] exists,up 4afa7113-1de4-4b0a-bb60-b349f2fef801 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:12.826 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:12.827 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:22:12.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:22:12.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:22:12.891 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:22:12.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:22:12.899 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:21:47.146+0000 7fcd33716a40 0 load: jerasure load: lrc 2026-02-24T17:22:12.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:22:12.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-24T17:22:12.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-24T17:22:12.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:22:12.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:22:12.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-24T17:22:12.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:22:12.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-24T17:22:12.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:22:12.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-24T17:22:13.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:22:13.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:22:13.537 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:22:13.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:22:14.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:22:14.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:22:14.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:22:14.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:22:14.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:22:14.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:22:14.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:22:14.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:22:14.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:22:14.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:22:14.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:22:14.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:22:14.648 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:22:14.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:22:14.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:22:14.649 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:22:14.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:22:14.862 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:14.863 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:22:14.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803784 2026-02-24T17:22:14.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803784 2026-02-24T17:22:14.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784' 2026-02-24T17:22:14.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:14.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:22:15.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574855 2026-02-24T17:22:15.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574855 2026-02-24T17:22:15.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574855' 2026-02-24T17:22:15.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:15.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:22:15.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411334 2026-02-24T17:22:15.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411334 2026-02-24T17:22:15.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-77309411334' 2026-02-24T17:22:15.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:15.102 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:22:15.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116998 2026-02-24T17:22:15.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116998 2026-02-24T17:22:15.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-77309411334 3-115964116998' 2026-02-24T17:22:15.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:15.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:22:15.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822661 2026-02-24T17:22:15.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822661 2026-02-24T17:22:15.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-77309411334 3-115964116998 4-154618822661' 2026-02-24T17:22:15.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:15.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:22:15.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=188978561028 2026-02-24T17:22:15.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 188978561028 2026-02-24T17:22:15.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-77309411334 3-115964116998 4-154618822661 5-188978561028' 2026-02-24T17:22:15.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:15.339 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:22:15.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332099 2026-02-24T17:22:15.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332099 2026-02-24T17:22:15.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-77309411334 3-115964116998 4-154618822661 5-188978561028 6-219043332099' 2026-02-24T17:22:15.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:15.420 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803784 2026-02-24T17:22:15.420 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:15.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:22:15.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803784 2026-02-24T17:22:15.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:15.422 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803784 2026-02-24T17:22:15.422 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-24T17:22:15.422 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803784 2026-02-24T17:22:15.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:22:15.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-24T17:22:15.640 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:22:16.641 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:22:16.641 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:22:16.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-24T17:22:16.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:16.861 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574855 2026-02-24T17:22:16.861 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:16.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:22:16.863 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574855 2026-02-24T17:22:16.863 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:16.864 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574855 2026-02-24T17:22:16.864 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-24T17:22:16.865 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 55834574855 2026-02-24T17:22:16.865 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:22:17.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-24T17:22:17.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:17.092 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411334 2026-02-24T17:22:17.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:17.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:22:17.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411334 2026-02-24T17:22:17.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:17.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411334 2026-02-24T17:22:17.096 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411334 2026-02-24T17:22:17.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-24T17:22:17.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:22:17.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-24T17:22:17.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964116998 2026-02-24T17:22:17.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:17.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:22:17.323 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964116998 2026-02-24T17:22:17.324 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:17.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116998 2026-02-24T17:22:17.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964116998' 2026-02-24T17:22:17.325 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 115964116998 2026-02-24T17:22:17.326 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:22:17.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964116999 -lt 115964116998 2026-02-24T17:22:17.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:17.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822661 2026-02-24T17:22:17.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:17.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:22:17.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822661 2026-02-24T17:22:17.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:17.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822661 2026-02-24T17:22:17.547 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 154618822661 2026-02-24T17:22:17.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822661' 2026-02-24T17:22:17.547 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:22:17.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822661 -lt 154618822661 2026-02-24T17:22:17.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:17.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-188978561028 2026-02-24T17:22:17.758 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:17.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:22:17.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-188978561028 2026-02-24T17:22:17.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:17.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=188978561028 2026-02-24T17:22:17.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-24T17:22:17.761 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 188978561028 2026-02-24T17:22:17.761 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:22:17.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-24T17:22:17.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:17.983 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-219043332099 2026-02-24T17:22:17.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:17.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:22:17.986 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-219043332099 2026-02-24T17:22:17.986 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:17.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332099 2026-02-24T17:22:17.987 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 219043332099 2026-02-24T17:22:17.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 219043332099' 2026-02-24T17:22:17.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:22:18.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332099 -lt 219043332099 2026-02-24T17:22:18.203 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:22:18.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:22:18.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:22:18.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:22:18.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:22:18.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:22:18.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:22:18.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:22:18.493 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:22:18.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:22:18.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:22:18.704 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:22:18.705 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:22:18.705 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:22:18.705 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:18.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:22:18.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:19.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-24T17:22:19.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:22:19.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:22:19.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:22:19.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:19.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:19.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:22:19.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:19.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-24T17:22:19.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:22:19.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:22:19.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:22:19.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:22:19.050 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:22:19.051 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:22:19.051 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:22:19.260 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:22:19.260 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-24T17:22:19.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:22:19.262 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-24T17:22:19.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:22:19.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-24T17:22:19.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:22:19.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-24T17:22:19.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:22:19.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:19.263 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:22:19.264 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:22:19.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:22:19.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:22:19.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:22:19.327 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-24T17:22:19.327 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:22:19.327 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:22:19.328 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:19.328 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:19.328 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:19.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:22:19.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:22:19.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-24T17:22:19.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-24T17:22:19.698 INFO:tasks.workunit.client.0.trial066.stderr:osd.0 is already out. 2026-02-24T17:22:19.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-24T17:22:19.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:22:19.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:22:19.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:22:19.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:22:19.709 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:22:19.709 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:22:19.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:22:19.709 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:22:19.710 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:22:19.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:22:19.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:22:19.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:22:19.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:22:19.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:22:19.818 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:20.031 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:22:20.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803787 2026-02-24T17:22:20.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803787 2026-02-24T17:22:20.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787' 2026-02-24T17:22:20.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:20.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:22:20.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574858 2026-02-24T17:22:20.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574858 2026-02-24T17:22:20.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-55834574858' 2026-02-24T17:22:20.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:20.192 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:22:20.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411337 2026-02-24T17:22:20.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411337 2026-02-24T17:22:20.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-77309411337' 2026-02-24T17:22:20.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:20.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:22:20.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964117001 2026-02-24T17:22:20.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964117001 2026-02-24T17:22:20.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-77309411337 3-115964117001' 2026-02-24T17:22:20.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:20.351 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:22:20.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=154618822664 2026-02-24T17:22:20.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 154618822664 2026-02-24T17:22:20.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-77309411337 3-115964117001 4-154618822664' 2026-02-24T17:22:20.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:20.429 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:22:20.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=188978561031 2026-02-24T17:22:20.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 188978561031 2026-02-24T17:22:20.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-77309411337 3-115964117001 4-154618822664 5-188978561031' 2026-02-24T17:22:20.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:20.509 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:22:20.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=219043332102 2026-02-24T17:22:20.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 219043332102 2026-02-24T17:22:20.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-55834574858 2-77309411337 3-115964117001 4-154618822664 5-188978561031 6-219043332102' 2026-02-24T17:22:20.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:20.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803787 2026-02-24T17:22:20.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:20.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:22:20.589 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803787 2026-02-24T17:22:20.589 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:20.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803787 2026-02-24T17:22:20.591 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803787 2026-02-24T17:22:20.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-24T17:22:20.591 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:22:20.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-24T17:22:20.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:20.810 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574858 2026-02-24T17:22:20.810 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:20.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:22:20.811 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574858 2026-02-24T17:22:20.812 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:20.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574858 2026-02-24T17:22:20.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-24T17:22:20.813 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 55834574858 2026-02-24T17:22:20.814 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:22:21.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-24T17:22:21.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:21.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411337 2026-02-24T17:22:21.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:21.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:22:21.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411337 2026-02-24T17:22:21.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:21.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411337 2026-02-24T17:22:21.031 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411337 2026-02-24T17:22:21.031 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411337' 2026-02-24T17:22:21.031 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:22:21.250 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-24T17:22:21.250 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:21.250 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964117001 2026-02-24T17:22:21.251 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:21.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:22:21.252 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964117001 2026-02-24T17:22:21.252 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:21.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964117001 2026-02-24T17:22:21.254 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 115964117001 2026-02-24T17:22:21.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964117001' 2026-02-24T17:22:21.254 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:22:21.472 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964117001 -lt 115964117001 2026-02-24T17:22:21.472 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:21.473 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-154618822664 2026-02-24T17:22:21.473 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:21.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:22:21.474 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-154618822664 2026-02-24T17:22:21.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:21.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=154618822664 2026-02-24T17:22:21.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 154618822664' 2026-02-24T17:22:21.476 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 154618822664 2026-02-24T17:22:21.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:22:21.689 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 154618822664 -lt 154618822664 2026-02-24T17:22:21.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:21.690 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-188978561031 2026-02-24T17:22:21.690 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:21.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:22:21.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-188978561031 2026-02-24T17:22:21.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:21.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=188978561031 2026-02-24T17:22:21.694 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 188978561031 2026-02-24T17:22:21.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 188978561031' 2026-02-24T17:22:21.694 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:22:21.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 188978561031 -lt 188978561031 2026-02-24T17:22:21.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:21.919 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-219043332102 2026-02-24T17:22:21.919 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:21.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:22:21.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-219043332102 2026-02-24T17:22:21.921 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:21.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=219043332102 2026-02-24T17:22:21.922 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 219043332102 2026-02-24T17:22:21.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 219043332102' 2026-02-24T17:22:21.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:22:22.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 219043332102 -lt 219043332102 2026-02-24T17:22:22.151 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:22:22.151 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:22:22.152 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:22:22.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:22:22.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:22:22.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:22:22.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:22:22.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:22:22.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:22:22.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:22:22.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:22:22.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:22:22.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:22:22.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:22:22.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:22:22.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:22:22.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:22:22.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-24T17:22:22.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:22:22.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:22:22.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-24T17:22:22.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:22:22.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:22:22.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-24T17:22:22.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:22:22.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:22:22.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-24T17:22:22.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:22:22.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:22:22.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-24T17:22:22.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:22:22.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:22:22.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-24T17:22:22.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:22:22.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:22:22.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-24T17:22:23.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:22:23.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:22:23.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:22:23.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:22:23.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:22:23.284 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:22:23.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:22:23.566 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:22:23.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:22:23.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:22:23.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:22:23.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:22:23.577 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:22:23.577 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:22:23.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:22:23.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:22:23.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:22:23.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:22:23.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:22:23.689 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:22:23.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:22:23.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:22:23.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:22:23.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:22:23.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:22:23.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:22:23.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:22:23.693 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:22:23.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:22:23.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:22:23.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:22:23.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:22:23.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:23.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:23.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:22:23.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:22:23.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:22:23.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:22:23.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:22:23.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:22:23.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:22:23.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:22:23.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:22:23.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:22:23.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:22:23.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:22:23.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:22:23.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:22:23.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:22:23.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:22:23.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:22:23.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:22:23.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:22:23.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:22:23.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:22:23.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:22:23.731 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:22:23.731 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:23.731 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:23.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:22:23.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:22:23.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:22:23.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:22:23.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:22:23.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:23.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:23.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:22:23.735 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:22:23.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:22:23.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:22:23.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:22:23.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:22:23.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:23.759 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:23.759 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:23.759 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:23.760 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:23.760 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:23.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:22:23.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:22:23.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:22:23.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:22:23.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:22:23.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:22:23.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:22:23.788 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:22:23.789 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:22:23.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:22:23.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:23.845 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:23.846 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:23.846 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:22:23.846 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:22:23.846 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:22:23.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:22:23.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:22:23.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:22:23.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:22:23.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:22:23.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:22:23.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:22:24.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:22:24.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:24.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:24.010 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:24.010 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:24.011 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:24.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:24.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:22:24.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:22:24.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:22:24.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:22:24.142 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:22:24.151 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:22:25.153 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:22:25.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:22:25.202 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:22:25.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:22:25.210 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:22:23.781+0000 7fce352d6ec0 0 load: jerasure load: lrc 2026-02-24T17:22:25.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:22:25.212 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:22:25.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:25.217 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:25.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:25.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:25.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:25.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:25.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:22:25.219 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:25.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6f8e4fc2-d3f6-45b3-9668-af4289417b35 2026-02-24T17:22:25.220 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 6f8e4fc2-d3f6-45b3-9668-af4289417b35 2026-02-24T17:22:25.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6f8e4fc2-d3f6-45b3-9668-af4289417b35' 2026-02-24T17:22:25.220 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:25.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBR3p1puotYDhAAwxOnHM6/4I7EsfkCIZv8xQ== 2026-02-24T17:22:25.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBR3p1puotYDhAAwxOnHM6/4I7EsfkCIZv8xQ=="}' 2026-02-24T17:22:25.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6f8e4fc2-d3f6-45b3-9668-af4289417b35 -i td/test-erasure-eio/0/new.json 2026-02-24T17:22:25.344 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:25.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:22:25.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBR3p1puotYDhAAwxOnHM6/4I7EsfkCIZv8xQ== --osd-uuid 6f8e4fc2-d3f6-45b3-9668-af4289417b35 2026-02-24T17:22:25.372 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:25.377+0000 7f7a71291a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:25.373 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:25.377+0000 7f7a71291a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:25.374 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:25.381+0000 7f7a71291a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:25.375 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:25.381+0000 7f7a71291a40 -1 bdev(0x562578dea400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:25.375 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:25.381+0000 7f7a71291a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:22:26.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:22:26.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:26.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:22:26.124 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:22:26.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:26.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:22:26.268 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:22:26.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:26.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:26.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:26.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:26.285 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:26.289+0000 7f226a797a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:26.287 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:26.293+0000 7f226a797a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:26.288 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:26.293+0000 7f226a797a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:22:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:22:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:26.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:26.386 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:26.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:26.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:26.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:22:26.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:26.709 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:26.713+0000 7f226a797a40 -1 Falling back to public interface 2026-02-24T17:22:27.256 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:27.261+0000 7f226a797a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:22:27.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:27.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:27.603 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:27.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:27.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:27.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:22:27.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:28.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:28.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:28.823 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:28.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:22:28.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:28.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:22:29.042 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2447496389,v1:127.0.0.1:6803/2447496389] [v2:127.0.0.1:6804/2447496389,v1:127.0.0.1:6805/2447496389] exists,up 6f8e4fc2-d3f6-45b3-9668-af4289417b35 2026-02-24T17:22:29.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:29.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:22:29.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:29.047 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:29.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:29.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:29.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:22:29.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:29.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8211261e-c80a-4689-a18d-38ac994cba7f 2026-02-24T17:22:29.050 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 8211261e-c80a-4689-a18d-38ac994cba7f 2026-02-24T17:22:29.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 8211261e-c80a-4689-a18d-38ac994cba7f' 2026-02-24T17:22:29.050 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:29.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBV3p1pejQrBBAA2GFCgGNq4FmMHO/CmolhkQ== 2026-02-24T17:22:29.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBV3p1pejQrBBAA2GFCgGNq4FmMHO/CmolhkQ=="}' 2026-02-24T17:22:29.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8211261e-c80a-4689-a18d-38ac994cba7f -i td/test-erasure-eio/1/new.json 2026-02-24T17:22:29.292 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:29.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:22:29.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBV3p1pejQrBBAA2GFCgGNq4FmMHO/CmolhkQ== --osd-uuid 8211261e-c80a-4689-a18d-38ac994cba7f 2026-02-24T17:22:29.323 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:29.329+0000 7f210833aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:29.325 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:29.329+0000 7f210833aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:29.326 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:29.329+0000 7f210833aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:29.326 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:29.329+0000 7f210833aa40 -1 bdev(0x564167c8a400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:29.327 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:29.329+0000 7f210833aa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:22:29.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:22:29.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:29.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:22:29.943 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:22:29.943 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:30.244 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:22:30.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:22:30.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:30.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:30.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:30.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:30.275 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:30.281+0000 7f6b32ba2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:30.276 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:30.281+0000 7f6b32ba2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:30.277 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:30.281+0000 7f6b32ba2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:30.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:22:30.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:30.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:22:30.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:30.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:30.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:30.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:30.462 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:30.463 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:30.463 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:22:30.668 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:30.673+0000 7f6b32ba2a40 -1 Falling back to public interface 2026-02-24T17:22:30.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:31.308 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:31.313+0000 7f6b32ba2a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:22:31.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:31.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:31.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:31.683 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:31.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:31.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:22:31.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:32.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:32.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:32.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:22:32.899 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:32.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:32.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:22:33.116 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2318975337,v1:127.0.0.1:6811/2318975337] [v2:127.0.0.1:6812/2318975337,v1:127.0.0.1:6813/2318975337] exists,up 8211261e-c80a-4689-a18d-38ac994cba7f 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:33.117 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:33.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:33.118 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:33.118 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:33.118 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:33.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:33.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:33.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:22:33.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:33.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b866fb87-b474-4987-87bb-37e08a1b8d3a 2026-02-24T17:22:33.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b866fb87-b474-4987-87bb-37e08a1b8d3a' 2026-02-24T17:22:33.122 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 b866fb87-b474-4987-87bb-37e08a1b8d3a 2026-02-24T17:22:33.122 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:33.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZ3p1p8EmsCBAAU+RKKfVQzYzj8xjY3murZw== 2026-02-24T17:22:33.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZ3p1p8EmsCBAAU+RKKfVQzYzj8xjY3murZw=="}' 2026-02-24T17:22:33.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b866fb87-b474-4987-87bb-37e08a1b8d3a -i td/test-erasure-eio/2/new.json 2026-02-24T17:22:33.360 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:33.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:22:33.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZ3p1p8EmsCBAAU+RKKfVQzYzj8xjY3murZw== --osd-uuid b866fb87-b474-4987-87bb-37e08a1b8d3a 2026-02-24T17:22:33.400 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:33.405+0000 7f555bd9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:33.401 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:33.405+0000 7f555bd9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:33.402 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:33.405+0000 7f555bd9da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:33.403 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:33.409+0000 7f555bd9da40 -1 bdev(0x55ed06d2a400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:33.403 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:33.409+0000 7f555bd9da40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:22:34.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:22:34.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:34.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:22:34.016 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:22:34.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:34.318 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:22:34.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:22:34.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:34.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:34.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:34.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:34.349 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:34.353+0000 7fae43572a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:34.351 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:34.357+0000 7fae43572a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:34.352 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:34.357+0000 7fae43572a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:34.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:22:34.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:34.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:22:34.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:34.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:34.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:34.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:34.544 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:34.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:34.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:22:34.712 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:34.717+0000 7fae43572a40 -1 Falling back to public interface 2026-02-24T17:22:34.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:35.292 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:35.297+0000 7fae43572a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:22:35.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:35.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:35.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:35.767 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:35.767 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:35.767 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:22:35.986 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3698182126,v1:127.0.0.1:6819/3698182126] [v2:127.0.0.1:6820/3698182126,v1:127.0.0.1:6821/3698182126] exists,up b866fb87-b474-4987-87bb-37e08a1b8d3a 2026-02-24T17:22:35.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:22:35.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:35.991 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:35.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:35.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:35.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:35.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:35.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:35.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:35.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:22:35.993 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:35.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5031172e-0e74-4190-a821-6de4131b6f7d 2026-02-24T17:22:35.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 5031172e-0e74-4190-a821-6de4131b6f7d' 2026-02-24T17:22:35.995 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 5031172e-0e74-4190-a821-6de4131b6f7d 2026-02-24T17:22:35.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:36.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBc3p1pecbyABAAij+0zavbKi4lrxTtxB7rVg== 2026-02-24T17:22:36.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBc3p1pecbyABAAij+0zavbKi4lrxTtxB7rVg=="}' 2026-02-24T17:22:36.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5031172e-0e74-4190-a821-6de4131b6f7d -i td/test-erasure-eio/3/new.json 2026-02-24T17:22:36.224 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:22:36.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:22:36.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBc3p1pecbyABAAij+0zavbKi4lrxTtxB7rVg== --osd-uuid 5031172e-0e74-4190-a821-6de4131b6f7d 2026-02-24T17:22:36.264 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:36.269+0000 7fc080d57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:36.265 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:36.269+0000 7fc080d57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:36.266 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:36.269+0000 7fc080d57a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:36.267 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:36.269+0000 7fc080d57a40 -1 bdev(0x560a88b84400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:36.267 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:36.273+0000 7fc080d57a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:22:36.937 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:22:36.937 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:36.938 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:22:36.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:22:36.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:37.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:22:37.232 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:22:37.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:37.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:37.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:37.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:37.262 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:37.265+0000 7f378255aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:37.264 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:37.269+0000 7f378255aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:37.265 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:37.269+0000 7f378255aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:37.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:22:37.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:37.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:22:37.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:37.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:37.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:37.448 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:37.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:37.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:37.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:22:37.640 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:37.645+0000 7f378255aa40 -1 Falling back to public interface 2026-02-24T17:22:37.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:38.252 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:38.257+0000 7f378255aa40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:22:38.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:38.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:38.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:38.668 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:38.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:38.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:22:38.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:39.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:39.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:39.899 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:39.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:22:39.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:39.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:22:40.123 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4028626290,v1:127.0.0.1:6827/4028626290] [v2:127.0.0.1:6828/4028626290,v1:127.0.0.1:6829/4028626290] exists,up 5031172e-0e74-4190-a821-6de4131b6f7d 2026-02-24T17:22:40.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:22:40.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:40.128 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:40.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:40.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:40.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:40.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:22:40.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:22:40.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-24T17:22:40.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:22:40.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=36774981-64cb-4df9-af2b-244d609236fd 2026-02-24T17:22:40.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 36774981-64cb-4df9-af2b-244d609236fd' 2026-02-24T17:22:40.131 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 36774981-64cb-4df9-af2b-244d609236fd 2026-02-24T17:22:40.131 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:22:40.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBg3p1pg7EOCRAAdcilmO6GP6GZBP4tERdVDQ== 2026-02-24T17:22:40.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBg3p1pg7EOCRAAdcilmO6GP6GZBP4tERdVDQ=="}' 2026-02-24T17:22:40.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 36774981-64cb-4df9-af2b-244d609236fd -i td/test-erasure-eio/4/new.json 2026-02-24T17:22:40.375 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:22:40.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-24T17:22:40.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBg3p1pg7EOCRAAdcilmO6GP6GZBP4tERdVDQ== --osd-uuid 36774981-64cb-4df9-af2b-244d609236fd 2026-02-24T17:22:40.405 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:40.409+0000 7fb51bc34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:40.406 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:40.409+0000 7fb51bc34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:40.407 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:40.413+0000 7fb51bc34a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:40.407 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:40.413+0000 7fb51bc34a40 -1 bdev(0x556582bd2400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:22:40.407 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:40.413+0000 7fb51bc34a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-24T17:22:41.037 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-24T17:22:41.037 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:22:41.038 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:22:41.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:22:41.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:22:41.339 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:22:41.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:22:41.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:41.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:22:41.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:22:41.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:22:41.368 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:41.373+0000 7f30812cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:41.370 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:41.373+0000 7f30812cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:41.371 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:41.377+0000 7f30812cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:41.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:22:41.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:41.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:22:41.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:41.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:41.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:41.558 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:41.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:41.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:41.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:22:41.732 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:41.737+0000 7f30812cfa40 -1 Falling back to public interface 2026-02-24T17:22:41.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:42.406 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:42.409+0000 7f30812cfa40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:22:42.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:42.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:42.779 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:42.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:42.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:42.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:22:42.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:43.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:43.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:43.996 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:22:43.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:22:43.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:43.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:22:44.216 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2052040359,v1:127.0.0.1:6835/2052040359] [v2:127.0.0.1:6836/2052040359,v1:127.0.0.1:6837/2052040359] exists,up 36774981-64cb-4df9-af2b-244d609236fd 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:44.218 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:22:44.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:22:44.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:22:44.282 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:22:44.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:22:44.290 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:22:26.717+0000 7f226a797a40 0 load: jerasure load: lrc 2026-02-24T17:22:44.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:22:44.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-24T17:22:44.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-24T17:22:44.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:22:44.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:22:44.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-24T17:22:44.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:22:44.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-24T17:22:44.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:22:44.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-24T17:22:44.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:22:44.570 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:22:44.929 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:22:44.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:22:45.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:22:45.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:22:45.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:22:45.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:22:45.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:22:45.942 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:22:45.943 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:22:45.943 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:22:45.943 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:22:45.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:22:46.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:22:46.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:22:46.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:22:46.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:22:46.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:22:46.050 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr:4' 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:46.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:22:46.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803782 2026-02-24T17:22:46.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803782 2026-02-24T17:22:46.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782' 2026-02-24T17:22:46.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:46.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:22:46.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607557 2026-02-24T17:22:46.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607557 2026-02-24T17:22:46.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557' 2026-02-24T17:22:46.429 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:46.429 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:22:46.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411334 2026-02-24T17:22:46.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411334 2026-02-24T17:22:46.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-77309411334' 2026-02-24T17:22:46.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:46.509 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:22:46.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182404 2026-02-24T17:22:46.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182404 2026-02-24T17:22:46.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-77309411334 3-107374182404' 2026-02-24T17:22:46.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:22:46.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:22:46.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=146028888067 2026-02-24T17:22:46.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 146028888067 2026-02-24T17:22:46.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-51539607557 2-77309411334 3-107374182404 4-146028888067' 2026-02-24T17:22:46.669 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:46.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803782 2026-02-24T17:22:46.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:46.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:22:46.671 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803782 2026-02-24T17:22:46.671 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:46.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803782 2026-02-24T17:22:46.673 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803782 2026-02-24T17:22:46.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-24T17:22:46.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:22:46.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-24T17:22:46.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:22:47.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:22:47.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:22:48.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-24T17:22:48.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:22:49.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:22:49.110 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:22:49.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-24T17:22:49.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:49.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607557 2026-02-24T17:22:49.328 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:49.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:22:49.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607557 2026-02-24T17:22:49.330 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:49.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607557 2026-02-24T17:22:49.332 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607557 2026-02-24T17:22:49.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-24T17:22:49.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:22:49.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607558 -lt 51539607557 2026-02-24T17:22:49.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:49.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411334 2026-02-24T17:22:49.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:49.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:22:49.547 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411334 2026-02-24T17:22:49.547 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:49.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411334 2026-02-24T17:22:49.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-24T17:22:49.549 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411334 2026-02-24T17:22:49.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:22:49.774 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-24T17:22:49.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:49.775 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182404 2026-02-24T17:22:49.775 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:49.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:22:49.777 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182404 2026-02-24T17:22:49.777 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:49.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182404 2026-02-24T17:22:49.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182404' 2026-02-24T17:22:49.779 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182404 2026-02-24T17:22:49.779 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:22:49.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182404 -lt 107374182404 2026-02-24T17:22:49.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:22:49.996 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-146028888067 2026-02-24T17:22:49.996 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:22:49.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:22:49.998 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-146028888067 2026-02-24T17:22:49.998 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:22:49.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=146028888067 2026-02-24T17:22:50.000 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 146028888067 2026-02-24T17:22:50.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 146028888067' 2026-02-24T17:22:50.000 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:22:50.210 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888067 -lt 146028888067 2026-02-24T17:22:50.210 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:22:50.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:22:50.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:22:50.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:22:50.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:22:50.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:22:50.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:22:50.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:22:50.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:22:50.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:22:50.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:22:50.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:22:50.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:22:50.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:22:50.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:22:50.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:22:50.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:22:50.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:22:50.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:46.206559+0000 0'0 42:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-24T17:22:44.885101+0000 0'0 2026-02-24T17:22:44.885101+0000 0 0 periodic scrub scheduled @ 2026-02-25T17:44:10.770784+0000 0 0 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.446581+0000 0'0 42:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:29:42.788869+0000 0 0 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.342705+0000 0'0 42:98 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:27:55.291271+0000 0 0 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.314660+0000 0'0 42:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-25T23:55:48.774899+0000 0 0 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:36.281032+0000 0'0 42:66 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:11:51.712461+0000 0 0 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout: 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T17:22:51.193 INFO:tasks.workunit.client.0.trial066.stderr:dumped pgs 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:22:51.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:51.226 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-24T17:22:51.226 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:22:51.226 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:22:51.226 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:22:51.226 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:22:51.451 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:22:51.452 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:22:51.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:22:51.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:22:51.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:22:51.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:22:51.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-24T17:22:51.971 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already down. 2026-02-24T17:22:51.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-24T17:22:52.225 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already out. 2026-02-24T17:22:52.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-24T17:22:52.437 INFO:tasks.workunit.client.0.trial066.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T17:22:52.437 INFO:tasks.workunit.client.0.trial066.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:46.206559+0000 0'0 42:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-24T17:22:44.885101+0000 0'0 2026-02-24T17:22:44.885101+0000 0 0 periodic scrub scheduled @ 2026-02-25T17:44:10.770784+0000 0 0 2026-02-24T17:22:52.437 INFO:tasks.workunit.client.0.trial066.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.446581+0000 0'0 42:22 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:29:42.788869+0000 0 0 2026-02-24T17:22:52.437 INFO:tasks.workunit.client.0.trial066.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.342705+0000 0'0 42:98 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:27:55.291271+0000 0 0 2026-02-24T17:22:52.438 INFO:tasks.workunit.client.0.trial066.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.314660+0000 0'0 42:22 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-25T23:55:48.774899+0000 0 0 2026-02-24T17:22:52.438 INFO:tasks.workunit.client.0.trial066.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:36.281032+0000 0'0 42:66 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T02:11:51.712461+0000 0 0 2026-02-24T17:22:52.438 INFO:tasks.workunit.client.0.trial066.stdout: 2026-02-24T17:22:52.438 INFO:tasks.workunit.client.0.trial066.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T17:22:52.438 INFO:tasks.workunit.client.0.trial066.stderr:dumped pgs 2026-02-24T17:22:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-24T17:22:52.448 INFO:tasks.workunit.client.0.trial066.stderr:4+0 records in 2026-02-24T17:22:52.448 INFO:tasks.workunit.client.0.trial066.stderr:4+0 records out 2026-02-24T17:22:52.448 INFO:tasks.workunit.client.0.trial066.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 7.1473e-05 s, 57.3 MB/s 2026-02-24T17:22:52.449 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-24T17:22:52.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.643 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.711 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.711 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:52.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:52.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:53.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:53.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.085 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.085 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.104 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.104 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.464 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.490 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.567 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.683 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.782 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:22:54.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-24T17:22:54.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj75 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T17:22:54.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:22:54.869 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj75 2026-02-24T17:22:54.869 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:22:54.869 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-24T17:22:54.869 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-24T17:22:54.869 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:22:55.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:22:55.079 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:22:55.079 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:22:55.079 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:22:55.079 INFO:tasks.workunit.client.0.trial066.stderr:2147483647' 2026-02-24T17:22:55.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-24T17:22:55.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-24T17:22:55.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:22:55.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-24T17:22:55.080 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:22:55.080 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-24T17:22:55.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:22:55.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-24T17:22:55.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:22:55.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-24T17:22:55.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:22:55.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:22:55.081 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:22:55.082 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-24T17:22:55.082 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:22:55.082 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:22:55.082 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:55.082 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:55.083 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:55.083 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:22:55.083 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:22:55.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:22:55.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:22:55.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:22:55.147 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-24T17:22:55.147 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:22:55.147 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:22:55.148 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:55.148 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:55.148 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:55.148 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:22:55.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:22:55.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-24T17:22:55.210 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-24T17:22:55.210 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:22:55.210 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:22:55.210 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj75 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj75 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:22:55.211 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-24T17:22:55.212 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-24T17:22:55.212 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:22:55.422 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:22:55.422 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:22:55.422 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr:2147483647' 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:22:55.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-24T17:22:55.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:22:55.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-24T17:22:55.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:22:55.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-24T17:22:55.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:22:55.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:22:55.425 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:22:55.425 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-24T17:22:55.425 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:22:55.425 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:22:55.425 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:55.426 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:55.426 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:55.426 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:22:55.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:22:55.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:22:55.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:22:55.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:22:55.488 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-24T17:22:55.488 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:22:55.488 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:22:55.489 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:22:55.489 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:55.489 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:55.489 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:22:55.489 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:22:55.489 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:22:55.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:22:55.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:22:55.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:22:55.554 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:22:55.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:22:55.555 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:22:55.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:22:55.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:22:55.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:22:55.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:22:55.576 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:55.581+0000 7fbeef1c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:55.577 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:55.581+0000 7fbeef1c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:55.578 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:55.581+0000 7fbeef1c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:22:55.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:22:55.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:22:55.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:22:55.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:22:55.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:22:55.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:55.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:22:55.771 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:22:55.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:55.771 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:22:55.972 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:55.977+0000 7fbeef1c6a40 -1 Falling back to public interface 2026-02-24T17:22:55.984 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:22:56.639 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:22:56.645+0000 7fbeef1c6a40 -1 osd.2 43 log_to_monitors true 2026-02-24T17:22:56.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:22:56.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:22:56.986 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:22:56.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:22:56.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:22:56.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:22:57.196 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up out weight 0 up_from 48 up_thru 19 down_at 43 last_clean_interval [18,43) [v2:127.0.0.1:6818/641438143,v1:127.0.0.1:6819/641438143] [v2:127.0.0.1:6820/641438143,v1:127.0.0.1:6821/641438143] exists,up b866fb87-b474-4987-87bb-37e08a1b8d3a 2026-02-24T17:22:57.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:22:57.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:22:57.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:22:57.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-24T17:22:57.444 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already in. 2026-02-24T17:22:57.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-24T17:23:12.455 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-24T17:23:12.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-24T17:23:12.457 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-24T17:23:12.457 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-24T17:23:12.457 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-24T17:23:12.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-24T17:23:12.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-24T17:23:12.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-24T17:23:12.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-24T17:23:12.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-24T17:23:12.664 INFO:tasks.workunit.client.0.trial066.stdout:active+recovery_unfound+degraded 2026-02-24T17:23:12.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-24T17:23:12.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-24T17:23:12.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-24T17:22:57.766090+0000 46'101 51:348 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-24T17:22:44.885101+0000 0'0 2026-02-24T17:22:44.885101+0000 0 0 no scrub is scheduled 0 0 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:57.650084+0000 0'0 51:58 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-25T23:27:11.946552+0000 0 0 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.342705+0000 0'0 51:115 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T03:27:55.291271+0000 0 0 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:43.314660+0000 0'0 51:40 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-25T23:55:48.774899+0000 0 0 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-24T17:22:57.652428+0000 0'0 51:102 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-24T17:22:24.132673+0000 0'0 2026-02-24T17:22:24.132673+0000 0 0 periodic scrub scheduled @ 2026-02-26T00:10:45.758467+0000 0 0 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout: 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-24T17:23:12.876 INFO:tasks.workunit.client.0.trial066.stderr:dumped pgs 2026-02-24T17:23:12.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout:{ 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "num_missing": 1, 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "num_unfound": 1, 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "objects": [ 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "oid": { 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "oid": "obj75", 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "key": "", 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "snapid": -2, 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "hash": 3565005477, 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "max": 0, 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "pool": 2, 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "namespace": "" 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "need": "46'76", 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "have": "0'0", 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "flags": "none", 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "locations": [ 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:23:12.972 INFO:tasks.workunit.client.0.trial066.stdout: ] 2026-02-24T17:23:12.973 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:12.973 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:12.973 INFO:tasks.workunit.client.0.trial066.stdout: "state": "NotRecovering", 2026-02-24T17:23:12.973 INFO:tasks.workunit.client.0.trial066.stdout: "available_might_have_unfound": true, 2026-02-24T17:23:12.973 INFO:tasks.workunit.client.0.trial066.stdout: "might_have_unfound": [], 2026-02-24T17:23:12.973 INFO:tasks.workunit.client.0.trial066.stdout: "more": false 2026-02-24T17:23:12.973 INFO:tasks.workunit.client.0.trial066.stdout:} 2026-02-24T17:23:12.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-24T17:23:13.056 INFO:tasks.workunit.client.0.trial066.stdout:{ 2026-02-24T17:23:13.056 INFO:tasks.workunit.client.0.trial066.stdout: "snap_trimq": "[]", 2026-02-24T17:23:13.056 INFO:tasks.workunit.client.0.trial066.stdout: "snap_trimq_len": 0, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+recovery_unfound+degraded", 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: "epoch": 51, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.057 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "acting_recovery_backfill": [ 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "2(4)", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "info": { 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s0", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "0", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "46'101", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "46'75", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "0'0", 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 101, 2026-02-24T17:23:13.058 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "50", 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 40, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 40, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 49, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 40, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 49, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 49, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 40, 2026-02-24T17:23:13.059 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "version": "46'101", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 349, 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 51, 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+recovery_unfound+degraded", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:22:57.766090+0000", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:22:57.766090+0000", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:22:57.766090+0000", 2026-02-24T17:23:13.060 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:22:57.766090+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:22:57.652800+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:22:57.652800+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:22:57.766090+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:22:57.502573+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:22:57.766090+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 49, 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "0'0", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "0'0", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "created": 40, 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.061 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 0, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:23:13.062 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 413696, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 101, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 505, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 1, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 3, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 1, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 101, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:23:13.063 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 101, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 404, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 99, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 405504, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:23:13.064 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.065 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.066 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 100 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),4(2)", 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 1 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:23:13.067 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "peer_info": [ 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "peer": "0(3)", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s3", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "3", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "46'101", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "46'101", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "0'0", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 101, 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:23:13.068 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 40, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 40, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 49, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 40, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 49, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 49, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 40, 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.069 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "version": "46'101", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 132, 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 46, 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+undersized+degraded", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.070 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:22:51.648370+0000", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:22:51.648221+0000", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 49, 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "0'0", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "0'0", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "created": 40, 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.071 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 0, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-25T23:44:50.068871+0000", 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:23:13.072 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 413696, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 101, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 505, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 101, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 101, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 101, 2026-02-24T17:23:13.073 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 404, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:23:13.074 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.075 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-24T17:23:13.076 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 101 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.077 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "peer": "1(1)", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s1", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "1", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "46'101", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "46'101", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "0'0", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 101, 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 40, 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 40, 2026-02-24T17:23:13.078 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 49, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 40, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 49, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 49, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 40, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:23:13.079 INFO:tasks.workunit.client.0.trial066.stdout: "version": "46'101", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 132, 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 46, 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+undersized+degraded", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:22:51.648370+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:22:51.648221+0000", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 49, 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "0'0", 2026-02-24T17:23:13.080 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "0'0", 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "created": 40, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 0, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:23:13.081 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-25T23:44:50.068871+0000", 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 413696, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 101, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 505, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T17:23:13.082 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 1, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 101, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 101, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 101, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 404, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:23:13.083 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:23:13.084 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:23:13.085 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 101 2026-02-24T17:23:13.086 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:23:13.087 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "peer": "2(4)", 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s4", 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "4", 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "46'101", 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "42'1", 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "0'0", 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 1, 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:23:13.088 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 40, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 40, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 49, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 40, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 49, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 49, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 40, 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.089 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "version": "42'1", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 18, 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 42, 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+clean", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:22:46.206559+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.090 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:22:46.206225+0000", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:22:46.206225+0000", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 49, 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "0'0", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "0'0", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "created": 40, 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.091 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 1, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 0, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 1, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-25T17:44:10.770784+0000", 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:23:13.092 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 4096, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 1, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 5, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 1, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 1, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 1, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 4, 2026-02-24T17:23:13.093 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:23:13.094 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.095 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [], 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [], 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.096 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "peer": "4(2)", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s2", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "shard": "2", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_update": "46'101", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_complete": "46'101", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "log_tail": "0'0", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_user_version": 101, 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill": "MAX", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete": [], 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [], 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "history": { 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_created": 40, 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "epoch_pool_created": 40, 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_started": 49, 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_interval_clean": 40, 2026-02-24T17:23:13.097 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_split": 0, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_marked_full": 0, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "same_up_since": 49, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "same_interval_since": 49, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "same_primary_since": 40, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "prior_readable_until_ub": 0 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "stats": { 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "version": "46'101", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "reported_seq": 132, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "reported_epoch": 46, 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "state": "active+undersized+degraded", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_fresh": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_change": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_active": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_peered": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.098 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean": "2026-02-24T17:22:51.233074+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_active": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_became_peered": "2026-02-24T17:22:51.649910+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_unstale": "2026-02-24T17:22:54.875091+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_undegraded": "2026-02-24T17:22:51.648370+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_fullsized": "2026-02-24T17:22:51.648221+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "mapping_epoch": 49, 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "log_start": "0'0", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_start": "0'0", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "created": 40, 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_clean": 41, 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "parent": "0.0", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "parent_split_bits": 0, 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub": "0'0", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub": "0'0", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_deep_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "last_clean_scrub_stamp": "2026-02-24T17:22:44.885101+0000", 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "objects_scrubbed": 0, 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "log_size": 101, 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "log_dups_size": 0, 2026-02-24T17:23:13.099 INFO:tasks.workunit.client.0.trial066.stdout: "ondisk_log_size": 101, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "stats_invalid": false, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "dirty_stats_invalid": false, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "omap_stats_invalid": false, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_stats_invalid": false, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "hitset_bytes_stats_invalid": false, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "pin_stats_invalid": false, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "manifest_stats_invalid": false, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrimq_len": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "last_scrub_duration": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-25T23:44:50.068871+0000", 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "scrub_duration": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "objects_trimmed": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "snaptrim_duration": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "stat_sum": { 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes": 413696, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects": 101, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_clones": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_object_copies": 505, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing_on_primary": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_missing": 0, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_degraded": 101, 2026-02-24T17:23:13.100 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_misplaced": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_unfound": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_dirty": 101, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_whiteouts": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_read": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_read_kb": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_write": 101, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_write_kb": 404, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_scrub_errors": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_shallow_scrub_errors": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_deep_scrub_errors": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_recovered": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_recovered": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_keys_recovered": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_omap": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_hit_set_archive": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_bytes_hit_set_archive": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_kb": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_kb": 0, 2026-02-24T17:23:13.101 INFO:tasks.workunit.client.0.trial066.stdout: "num_promote": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_high": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_flush_mode_low": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_some": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_evict_mode_full": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_pinned": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_legacy_snapsets": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_large_omap_objects": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_manifest": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_bytes": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_omap_keys": 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "num_objects_repaired": 0 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "up": [ 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.102 INFO:tasks.workunit.client.0.trial066.stdout: "acting": [ 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: 3, 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: 1, 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: 4, 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: 0, 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: 2 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "avail_no_missing": [ 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "3(0)", 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "0(3)", 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "1(1)", 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "4(2)" 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "object_location_counts": [ 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "objects": 101 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "blocked_by": [], 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "up_primary": 3, 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "acting_primary": 3, 2026-02-24T17:23:13.103 INFO:tasks.workunit.client.0.trial066.stdout: "purged_snaps": [] 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "empty": 0, 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "dne": 0, 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "incomplete": 0, 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "last_epoch_started": 50, 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "hit_set_history": { 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "current_last_update": "0'0", 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "history": [] 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "recovery_state": [ 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "name": "Started/Primary/Active", 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "enter_time": "2026-02-24T17:22:57.502065+0000", 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "might_have_unfound": [ 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "0(3)", 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "status": "already probed" 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "1(1)", 2026-02-24T17:23:13.104 INFO:tasks.workunit.client.0.trial066.stdout: "status": "already probed" 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "2(4)", 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "status": "already probed" 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "osd": "4(2)", 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "status": "already probed" 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "recovery_progress": { 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "backfill_targets": [], 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "waiting_on_backfill": [], 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "last_backfill_started": "MIN", 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "backfill_info": { 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "begin": "MIN", 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "end": "MIN", 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "objects": [] 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "peer_backfill_info": [], 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "backfills_in_flight": [], 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "recovering": [], 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "pg_backend": { 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "recovery_ops": [], 2026-02-24T17:23:13.105 INFO:tasks.workunit.client.0.trial066.stdout: "read_ops": [] 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: { 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "name": "Started", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "enter_time": "2026-02-24T17:22:57.399872+0000" 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: ], 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "scrubber": { 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "active": false, 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "must_scrub": false, 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "must_deep_scrub": false, 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "must_repair": false, 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "sched_time": "2026-02-25T23:44:50.068871+0000", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "schedule": "not registered for scrubbing", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "urgency": "periodic-regular", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "shallow-target": { 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s0", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "level": "shallow", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "urgency": "periodic-regular", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "sched_time": "2026-02-25T23:44:50.068871+0000", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "orig_sched_time": "2026-02-25T23:44:50.068871+0000", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "last_issue": "ok", 2026-02-24T17:23:13.106 INFO:tasks.workunit.client.0.trial066.stdout: "forced": false, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "eligible": false, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "queued": false, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "active": false 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "deep-target": { 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "pgid": "2.0s0", 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "level": "deep", 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "urgency": "periodic-regular", 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "sched_time": "2026-03-02T04:58:59.915400+0000", 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "orig_sched_time": "2026-03-02T04:58:59.915400+0000", 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "last_issue": "ok", 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "forced": false, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "eligible": false, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "queued": false, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "active": false 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: } 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: }, 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout: "agent_state": {} 2026-02-24T17:23:13.107 INFO:tasks.workunit.client.0.trial066.stdout:} 2026-02-24T17:23:13.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:23:13.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-24T17:23:13.156 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:23:13.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-24T17:23:13.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-24T17:23:13.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-24T17:23:13.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-24T17:23:13.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-24T17:23:13.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-24T17:23:13.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-24T17:23:13.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-24T17:23:18.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-24T17:23:18.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-24T17:23:18.390 INFO:tasks.workunit.client.0.trial066.stderr:pg has no unfound objects 2026-02-24T17:23:18.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-24T17:23:18.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:23:18.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:23:18.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:23:18.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:23:18.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:23:18.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:23:18.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:23:18.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:23:18.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:23:18.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:23:18.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:23:18.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:23:18.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:23:18.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:23:18.508 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr:4' 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:18.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:23:18.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803791 2026-02-24T17:23:18.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803791 2026-02-24T17:23:18.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803791' 2026-02-24T17:23:18.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:18.800 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:23:18.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607566 2026-02-24T17:23:18.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607566 2026-02-24T17:23:18.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803791 1-51539607566' 2026-02-24T17:23:18.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:18.878 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:23:18.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430215 2026-02-24T17:23:18.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430215 2026-02-24T17:23:18.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803791 1-51539607566 2-206158430215' 2026-02-24T17:23:18.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:18.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:23:19.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182412 2026-02-24T17:23:19.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182412 2026-02-24T17:23:19.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803791 1-51539607566 2-206158430215 3-107374182412' 2026-02-24T17:23:19.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:19.036 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:23:19.116 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=146028888076 2026-02-24T17:23:19.116 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 146028888076 2026-02-24T17:23:19.116 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803791 1-51539607566 2-206158430215 3-107374182412 4-146028888076' 2026-02-24T17:23:19.116 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:19.117 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803791 2026-02-24T17:23:19.117 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:19.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:23:19.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803791 2026-02-24T17:23:19.118 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:19.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803791 2026-02-24T17:23:19.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-24T17:23:19.120 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803791 2026-02-24T17:23:19.120 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:23:19.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803788 -lt 25769803791 2026-02-24T17:23:19.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:23:20.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:23:20.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:23:20.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-24T17:23:20.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:20.547 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607566 2026-02-24T17:23:20.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:20.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:23:20.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607566 2026-02-24T17:23:20.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:20.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607566 2026-02-24T17:23:20.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607566' 2026-02-24T17:23:20.551 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607566 2026-02-24T17:23:20.552 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:23:20.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607566 -lt 51539607566 2026-02-24T17:23:20.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:20.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-206158430215 2026-02-24T17:23:20.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:20.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:23:20.765 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-206158430215 2026-02-24T17:23:20.765 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:20.767 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430215 2026-02-24T17:23:20.767 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 206158430215 2026-02-24T17:23:20.767 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 206158430215' 2026-02-24T17:23:20.767 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:23:20.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430215 -lt 206158430215 2026-02-24T17:23:20.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:20.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182412 2026-02-24T17:23:20.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:20.983 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:23:20.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182412 2026-02-24T17:23:20.984 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:20.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182412 2026-02-24T17:23:20.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182412' 2026-02-24T17:23:20.985 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182412 2026-02-24T17:23:20.986 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:23:21.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182413 -lt 107374182412 2026-02-24T17:23:21.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:21.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-146028888076 2026-02-24T17:23:21.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:21.210 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:23:21.210 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-146028888076 2026-02-24T17:23:21.210 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:21.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=146028888076 2026-02-24T17:23:21.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 146028888076' 2026-02-24T17:23:21.212 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 146028888076 2026-02-24T17:23:21.212 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:23:21.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888076 -lt 146028888076 2026-02-24T17:23:21.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:23:21.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:23:21.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:23:21.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:23:21.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:23:21.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:23:21.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:23:21.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:23:21.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:23:21.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:23:21.716 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:23:21.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:23:21.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:23:21.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:23:21.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:23:22.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:23:22.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:23:22.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:23:22.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-24T17:23:22.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-24T17:23:22.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-24T17:23:22.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.247 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-24T17:23:22.247 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-24T17:23:22.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-24T17:23:22.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-24T17:23:22.314 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-24T17:23:22.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-24T17:23:22.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-24T17:23:22.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-24T17:23:22.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-24T17:23:22.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-24T17:23:22.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-24T17:23:22.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.489 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-24T17:23:22.491 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-24T17:23:22.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-24T17:23:22.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-24T17:23:22.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.575 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-24T17:23:22.577 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-24T17:23:22.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-24T17:23:22.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-24T17:23:22.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-24T17:23:22.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.689 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-24T17:23:22.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-24T17:23:22.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-24T17:23:22.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.758 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-24T17:23:22.760 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.783 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-24T17:23:22.784 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-24T17:23:22.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.824 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-24T17:23:22.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-24T17:23:22.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-24T17:23:22.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-24T17:23:22.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-24T17:23:22.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-24T17:23:22.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-24T17:23:22.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:22.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:22.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-24T17:23:22.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-24T17:23:23.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-24T17:23:23.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-24T17:23:23.090 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-24T17:23:23.112 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-24T17:23:23.135 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-24T17:23:23.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-24T17:23:23.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-24T17:23:23.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-24T17:23:23.226 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.249 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.250 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-24T17:23:23.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-24T17:23:23.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-24T17:23:23.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-24T17:23:23.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-24T17:23:23.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-24T17:23:23.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-24T17:23:23.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-24T17:23:23.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-24T17:23:23.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-24T17:23:23.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-24T17:23:23.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-24T17:23:23.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-24T17:23:23.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-24T17:23:23.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-24T17:23:23.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-24T17:23:23.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-24T17:23:23.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-24T17:23:23.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-24T17:23:23.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-24T17:23:23.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-24T17:23:23.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-24T17:23:23.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-24T17:23:23.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.767 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-24T17:23:23.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.790 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.790 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-24T17:23:23.790 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-24T17:23:23.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-24T17:23:23.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-24T17:23:23.862 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.877 INFO:tasks.workunit.client.0.trial066.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-24T17:23:23.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-24T17:23:23.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-24T17:23:23.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-24T17:23:23.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.956 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-24T17:23:23.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-24T17:23:23.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:23.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:23.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-24T17:23:23.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.004 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-24T17:23:24.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.029 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-24T17:23:24.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-24T17:23:24.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-24T17:23:24.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-24T17:23:24.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-24T17:23:24.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-24T17:23:24.150 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-24T17:23:24.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-24T17:23:24.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-24T17:23:24.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-24T17:23:24.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-24T17:23:24.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-24T17:23:24.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-24T17:23:24.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-24T17:23:24.326 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-24T17:23:24.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-24T17:23:24.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-24T17:23:24.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-24T17:23:24.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-24T17:23:24.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-24T17:23:24.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-24T17:23:24.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-24T17:23:24.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:23:24.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:23:24.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:23:24.738 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:23:24.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:23:25.008 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:23:25.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:23:25.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:23:25.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:23:25.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:23:25.018 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:23:25.018 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:23:25.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:23:25.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:23:25.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:23:25.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:23:25.329 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:23:25.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:23:25.331 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:23:25.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:23:25.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:23:25.333 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:23:25.334 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:23:25.334 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:23:25.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:23:25.335 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:23:25.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:23:25.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:23:25.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:23:25.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:23:25.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:25.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:25.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:23:25.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:23:25.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:23:25.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:23:25.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:23:25.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:23:25.370 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:23:25.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:23:25.371 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:23:25.373 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:23:25.373 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:23:25.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:23:25.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:23:25.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:23:25.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:23:25.375 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:23:25.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:23:25.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:23:25.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:23:25.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:23:25.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:25.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:25.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:23:25.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:23:25.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:23:25.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:23:25.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:23:25.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:25.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:25.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:23:25.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:23:25.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:23:25.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:23:25.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:23:25.406 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:23:25.407 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:25.407 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:25.407 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:25.407 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:25.407 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:25.407 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:25.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:23:25.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:23:25.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:23:25.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:23:25.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:23:25.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:23:25.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:23:25.438 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:23:25.438 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:23:25.438 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:23:25.438 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:23:25.438 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:25.438 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:25.438 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:23:25.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:23:25.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:23:25.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:23:25.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:23:25.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:23:25.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:23:25.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:23:25.495 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:23:25.495 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:23:25.495 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:23:25.496 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:23:25.496 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:25.496 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:25.496 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:23:25.496 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:23:25.496 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:23:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:23:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:23:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:23:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:23:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:23:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:23:25.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:23:25.662 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:23:25.662 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:25.662 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:25.663 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:25.663 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:25.663 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:25.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:25.664 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:23:25.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:23:25.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:23:25.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:23:25.790 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:23:25.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:23:26.800 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:23:26.851 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:23:26.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:23:26.859 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:23:25.432+0000 7f2b8c525ec0 0 load: jerasure load: lrc 2026-02-24T17:23:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-24T17:23:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-24T17:23:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-24T17:23:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-24T17:23:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-24T17:23:26.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:23:26.861 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-24T17:23:26.861 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-24T17:23:26.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:23:26.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:23:26.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:23:26.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:23:26.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:23:26.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:23:26.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:23:26.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:23:26.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:23:26.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:23:26.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:23:26.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:23:26.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:23:26.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:23:26.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:23:26.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:23:26.871 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:23:26.871 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:26.871 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:26.871 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:26.871 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:26.871 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:26.872 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:26.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:26.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:23:26.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:23:26.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:23:26.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:23:26.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:23:26.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:23:26.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:23:26.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:23:26.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:23:26.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:23:26.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:23:26.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:23:26.874 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:23:26.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fabe1c6e-9a56-4b1b-a9d1-c6fd57a9af42 2026-02-24T17:23:26.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fabe1c6e-9a56-4b1b-a9d1-c6fd57a9af42' 2026-02-24T17:23:26.875 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 fabe1c6e-9a56-4b1b-a9d1-c6fd57a9af42 2026-02-24T17:23:26.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:23:26.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCO3p1pl/3GNBAAm3VxwjurcyPNFuFsB73O6w== 2026-02-24T17:23:26.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCO3p1pl/3GNBAAm3VxwjurcyPNFuFsB73O6w=="}' 2026-02-24T17:23:26.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fabe1c6e-9a56-4b1b-a9d1-c6fd57a9af42 -i td/test-erasure-eio/0/new.json 2026-02-24T17:23:26.989 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:26.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:23:26.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCO3p1pl/3GNBAAm3VxwjurcyPNFuFsB73O6w== --osd-uuid fabe1c6e-9a56-4b1b-a9d1-c6fd57a9af42 2026-02-24T17:23:27.014 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.020+0000 7f2f56393a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:27.016 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.024+0000 7f2f56393a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:27.017 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.024+0000 7f2f56393a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:27.017 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.024+0000 7f2f56393a40 -1 bdev(0x5572d2870400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:23:27.017 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.024+0000 7f2f56393a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:23:27.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:23:27.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:23:27.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:23:27.702 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:23:27.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:23:27.852 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:23:27.853 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:23:27.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:23:27.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:23:27.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:23:27.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:23:27.867 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.876+0000 7fac39ecba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:27.869 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.876+0000 7fac39ecba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:27.870 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:27.876+0000 7fac39ecba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:27.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:23:27.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:23:27.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:23:27.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:23:27.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:23:27.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:27.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:23:27.966 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:27.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:27.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:23:28.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:28.316 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:28.324+0000 7fac39ecba40 -1 Falling back to public interface 2026-02-24T17:23:29.003 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:29.008+0000 7fac39ecba40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:23:29.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:29.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:29.180 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:29.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:23:29.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:29.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:23:29.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:30.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:30.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:30.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:23:30.395 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:23:30.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:30.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:23:30.612 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/483929765,v1:127.0.0.1:6803/483929765] [v2:127.0.0.1:6804/483929765,v1:127.0.0.1:6805/483929765] exists,up fabe1c6e-9a56-4b1b-a9d1-c6fd57a9af42 2026-02-24T17:23:30.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:23:30.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:23:30.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:23:30.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:23:30.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:23:30.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:30.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:23:30.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:23:30.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:23:30.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a8891944-2b31-4977-9291-00b95569ab5f 2026-02-24T17:23:30.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 a8891944-2b31-4977-9291-00b95569ab5f' 2026-02-24T17:23:30.621 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 a8891944-2b31-4977-9291-00b95569ab5f 2026-02-24T17:23:30.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:23:30.629 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCS3p1pskEUJhAAi0m7kAjPLrOtA0j8sBnDAg== 2026-02-24T17:23:30.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCS3p1pskEUJhAAi0m7kAjPLrOtA0j8sBnDAg=="}' 2026-02-24T17:23:30.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a8891944-2b31-4977-9291-00b95569ab5f -i td/test-erasure-eio/1/new.json 2026-02-24T17:23:30.849 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:30.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:23:30.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCS3p1pskEUJhAAi0m7kAjPLrOtA0j8sBnDAg== --osd-uuid a8891944-2b31-4977-9291-00b95569ab5f 2026-02-24T17:23:30.876 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:30.884+0000 7f0f937b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:30.878 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:30.884+0000 7f0f937b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:30.879 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:30.884+0000 7f0f937b2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:30.880 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:30.888+0000 7f0f937b2a40 -1 bdev(0x55f928476400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:23:30.880 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:30.888+0000 7f0f937b2a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:23:31.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:23:31.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:23:31.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:23:31.511 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:23:31.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:23:31.810 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:23:31.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:23:31.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:23:31.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:23:31.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:23:31.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:23:31.826 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:31.832+0000 7f93b462fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:31.827 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:31.836+0000 7f93b462fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:31.828 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:31.836+0000 7f93b462fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:32.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:23:32.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:23:32.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:23:32.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:23:32.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:23:32.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:32.021 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:32.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:23:32.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:32.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:23:32.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:32.345 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:32.352+0000 7f93b462fa40 -1 Falling back to public interface 2026-02-24T17:23:32.961 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:32.968+0000 7f93b462fa40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:23:33.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:33.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:33.228 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:33.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:23:33.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:33.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:23:33.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:34.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:34.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:34.439 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:23:34.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:23:34.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:34.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:23:34.649 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3068492012,v1:127.0.0.1:6811/3068492012] [v2:127.0.0.1:6812/3068492012,v1:127.0.0.1:6813/3068492012] exists,up a8891944-2b31-4977-9291-00b95569ab5f 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:23:34.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:34.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:23:34.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:23:34.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:23:34.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:23:34.654 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a57848cc-84bc-4dc0-afe5-791ae52b60dd 2026-02-24T17:23:34.654 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 a57848cc-84bc-4dc0-afe5-791ae52b60dd' 2026-02-24T17:23:34.654 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 a57848cc-84bc-4dc0-afe5-791ae52b60dd 2026-02-24T17:23:34.654 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:23:34.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCW3p1pS7k+KBAARCAMITS6CqsaIJk4AQAmDA== 2026-02-24T17:23:34.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCW3p1pS7k+KBAARCAMITS6CqsaIJk4AQAmDA=="}' 2026-02-24T17:23:34.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a57848cc-84bc-4dc0-afe5-791ae52b60dd -i td/test-erasure-eio/2/new.json 2026-02-24T17:23:34.887 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:23:34.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:23:34.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCW3p1pS7k+KBAARCAMITS6CqsaIJk4AQAmDA== --osd-uuid a57848cc-84bc-4dc0-afe5-791ae52b60dd 2026-02-24T17:23:34.924 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:34.932+0000 7efe49b29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:34.926 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:34.932+0000 7efe49b29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:34.927 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:34.936+0000 7efe49b29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:34.928 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:34.936+0000 7efe49b29a40 -1 bdev(0x55edf4858400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:23:34.928 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:34.936+0000 7efe49b29a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:23:35.582 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:23:35.582 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:23:35.583 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:23:35.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:23:35.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:23:35.873 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:23:35.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:23:35.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:23:35.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:23:35.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:23:35.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:23:35.898 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:35.904+0000 7fb335452a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:35.899 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:35.908+0000 7fb335452a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:35.901 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:35.908+0000 7fb335452a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:36.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:23:36.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:23:36.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:23:36.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:23:36.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:23:36.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:36.088 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:23:36.089 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:36.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:36.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:23:36.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:36.333 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:36.340+0000 7fb335452a40 -1 Falling back to public interface 2026-02-24T17:23:36.909 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:36.916+0000 7fb335452a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:23:37.312 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:37.312 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:37.312 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:37.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:23:37.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:37.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/913604170,v1:127.0.0.1:6819/913604170] [v2:127.0.0.1:6820/913604170,v1:127.0.0.1:6821/913604170] exists,up a57848cc-84bc-4dc0-afe5-791ae52b60dd 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:23:37.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:37.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:23:37.540 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:23:37.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:23:37.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2fdb0b9b-4a9e-4a3a-9466-7082bbb9aad7 2026-02-24T17:23:37.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2fdb0b9b-4a9e-4a3a-9466-7082bbb9aad7' 2026-02-24T17:23:37.542 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 2fdb0b9b-4a9e-4a3a-9466-7082bbb9aad7 2026-02-24T17:23:37.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:23:37.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZ3p1pnkicIRAAZvFoy4b3EvHVoBKKngqqkw== 2026-02-24T17:23:37.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZ3p1pnkicIRAAZvFoy4b3EvHVoBKKngqqkw=="}' 2026-02-24T17:23:37.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2fdb0b9b-4a9e-4a3a-9466-7082bbb9aad7 -i td/test-erasure-eio/3/new.json 2026-02-24T17:23:37.768 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:23:37.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:23:37.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZ3p1pnkicIRAAZvFoy4b3EvHVoBKKngqqkw== --osd-uuid 2fdb0b9b-4a9e-4a3a-9466-7082bbb9aad7 2026-02-24T17:23:37.796 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:37.804+0000 7f1bc81daa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:37.797 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:37.804+0000 7f1bc81daa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:37.799 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:37.804+0000 7f1bc81daa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:37.799 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:37.808+0000 7f1bc81daa40 -1 bdev(0x564d933c2400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:23:37.799 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:37.808+0000 7f1bc81daa40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:23:38.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:23:38.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:23:38.472 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:23:38.472 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:23:38.472 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:23:38.760 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:23:38.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:23:38.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:23:38.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:23:38.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:23:38.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:23:38.784 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:38.792+0000 7f8c85b73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:38.786 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:38.792+0000 7f8c85b73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:38.787 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:38.796+0000 7f8c85b73a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:38.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:23:38.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:23:38.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:23:38.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:23:38.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:23:38.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:38.976 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:38.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:23:38.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:38.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:23:39.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:39.257 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:39.264+0000 7f8c85b73a40 -1 Falling back to public interface 2026-02-24T17:23:39.796 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:39.804+0000 7f8c85b73a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:23:40.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:40.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:40.188 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:40.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:23:40.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:40.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:23:40.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:41.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:41.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:41.402 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:23:41.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:23:41.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:41.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:23:41.616 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1964450721,v1:127.0.0.1:6827/1964450721] [v2:127.0.0.1:6828/1964450721,v1:127.0.0.1:6829/1964450721] exists,up 2fdb0b9b-4a9e-4a3a-9466-7082bbb9aad7 2026-02-24T17:23:41.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:23:41.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:23:41.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:23:41.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:23:41.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:41.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:23:41.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:23:41.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:23:41.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-24T17:23:41.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:23:41.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7d3f1783-0ed1-406e-9ab9-c85635c68ef4 2026-02-24T17:23:41.624 INFO:tasks.workunit.client.0.trial066.stdout:add osd4 7d3f1783-0ed1-406e-9ab9-c85635c68ef4 2026-02-24T17:23:41.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 7d3f1783-0ed1-406e-9ab9-c85635c68ef4' 2026-02-24T17:23:41.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:23:41.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCd3p1p629QJhAA1TDFKYYdbjrkvEsyTFcr5A== 2026-02-24T17:23:41.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCd3p1p629QJhAA1TDFKYYdbjrkvEsyTFcr5A=="}' 2026-02-24T17:23:41.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7d3f1783-0ed1-406e-9ab9-c85635c68ef4 -i td/test-erasure-eio/4/new.json 2026-02-24T17:23:41.850 INFO:tasks.workunit.client.0.trial066.stdout:4 2026-02-24T17:23:41.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-24T17:23:41.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCd3p1p629QJhAA1TDFKYYdbjrkvEsyTFcr5A== --osd-uuid 7d3f1783-0ed1-406e-9ab9-c85635c68ef4 2026-02-24T17:23:41.877 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:41.884+0000 7f0b93e18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:41.878 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:41.884+0000 7f0b93e18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:41.879 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:41.884+0000 7f0b93e18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:41.880 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:41.888+0000 7f0b93e18a40 -1 bdev(0x556c1f1fa400 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-24T17:23:41.880 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:41.888+0000 7f0b93e18a40 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-24T17:23:42.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-24T17:23:42.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:23:42.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-24T17:23:42.511 INFO:tasks.workunit.client.0.trial066.stdout:adding osd4 key to auth repository 2026-02-24T17:23:42.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:23:42.801 INFO:tasks.workunit.client.0.trial066.stdout:start osd.4 2026-02-24T17:23:42.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-24T17:23:42.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:23:42.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:23:42.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:23:42.801 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:23:42.825 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:42.832+0000 7fe02c726a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:42.827 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:42.832+0000 7fe02c726a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:42.828 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:42.836+0000 7fe02c726a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:43.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-24T17:23:43.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:23:43.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-24T17:23:43.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:23:43.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:23:43.019 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:43.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:43.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:23:43.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:43.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:23:43.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:43.264 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:43.272+0000 7fe02c726a40 -1 Falling back to public interface 2026-02-24T17:23:43.910 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:43.916+0000 7fe02c726a40 -1 osd.4 0 log_to_monitors true 2026-02-24T17:23:44.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:44.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:44.232 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:44.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:23:44.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:44.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:23:44.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:45.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:45.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:45.448 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:23:45.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:23:45.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:45.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-24T17:23:45.656 INFO:tasks.workunit.client.0.trial066.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1730543704,v1:127.0.0.1:6835/1730543704] [v2:127.0.0.1:6836/1730543704,v1:127.0.0.1:6837/1730543704] exists,up 7d3f1783-0ed1-406e-9ab9-c85635c68ef4 2026-02-24T17:23:45.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:23:45.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:45.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:23:45.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:23:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:23:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:23:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:23:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-24T17:23:45.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:23:45.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0a7ad994-ac22-4e2b-912f-96a3b87e5fb1 2026-02-24T17:23:45.664 INFO:tasks.workunit.client.0.trial066.stdout:add osd5 0a7ad994-ac22-4e2b-912f-96a3b87e5fb1 2026-02-24T17:23:45.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 0a7ad994-ac22-4e2b-912f-96a3b87e5fb1' 2026-02-24T17:23:45.664 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:23:45.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCh3p1pVWymKBAAJn/Fd/Q2r9ba8p8wrFcGWQ== 2026-02-24T17:23:45.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCh3p1pVWymKBAAJn/Fd/Q2r9ba8p8wrFcGWQ=="}' 2026-02-24T17:23:45.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0a7ad994-ac22-4e2b-912f-96a3b87e5fb1 -i td/test-erasure-eio/5/new.json 2026-02-24T17:23:45.890 INFO:tasks.workunit.client.0.trial066.stdout:5 2026-02-24T17:23:45.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-24T17:23:45.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCh3p1pVWymKBAAJn/Fd/Q2r9ba8p8wrFcGWQ== --osd-uuid 0a7ad994-ac22-4e2b-912f-96a3b87e5fb1 2026-02-24T17:23:45.925 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:45.932+0000 7f54d29b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:45.927 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:45.932+0000 7f54d29b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:45.928 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:45.936+0000 7f54d29b7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:45.928 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:45.936+0000 7f54d29b7a40 -1 bdev(0x561393576400 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-24T17:23:45.929 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:45.936+0000 7f54d29b7a40 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-24T17:23:46.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-24T17:23:46.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:23:46.555 INFO:tasks.workunit.client.0.trial066.stdout:adding osd5 key to auth repository 2026-02-24T17:23:46.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-24T17:23:46.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:23:46.841 INFO:tasks.workunit.client.0.trial066.stdout:start osd.5 2026-02-24T17:23:46.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-24T17:23:46.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:23:46.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:23:46.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:23:46.842 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:23:46.866 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:46.872+0000 7f8b8d8a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:46.868 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:46.876+0000 7f8b8d8a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:46.869 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:46.876+0000 7f8b8d8a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:47.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-24T17:23:47.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:23:47.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-24T17:23:47.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:23:47.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:23:47.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:47.063 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:47.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:23:47.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:47.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:23:47.241 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:47.248+0000 7f8b8d8a8a40 -1 Falling back to public interface 2026-02-24T17:23:47.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:47.962 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:47.968+0000 7f8b8d8a8a40 -1 osd.5 0 log_to_monitors true 2026-02-24T17:23:48.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:48.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:48.272 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:48.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:23:48.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:48.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:23:48.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:49.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:49.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:49.484 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:23:49.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:23:49.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:49.485 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/1224250326,v1:127.0.0.1:6843/1224250326] [v2:127.0.0.1:6844/1224250326,v1:127.0.0.1:6845/1224250326] exists,up 0a7ad994-ac22-4e2b-912f-96a3b87e5fb1 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-24T17:23:49.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:49.692 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:23:49.693 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-24T17:23:49.694 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:23:49.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2be4f51b-5ca1-4993-8eb9-37a1ec6e6ed7 2026-02-24T17:23:49.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 2be4f51b-5ca1-4993-8eb9-37a1ec6e6ed7' 2026-02-24T17:23:49.695 INFO:tasks.workunit.client.0.trial066.stdout:add osd6 2be4f51b-5ca1-4993-8eb9-37a1ec6e6ed7 2026-02-24T17:23:49.695 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:23:49.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCl3p1pqF/JKhAAvsuC4Y+aV57MXy3D8TgBng== 2026-02-24T17:23:49.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCl3p1pqF/JKhAAvsuC4Y+aV57MXy3D8TgBng=="}' 2026-02-24T17:23:49.709 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2be4f51b-5ca1-4993-8eb9-37a1ec6e6ed7 -i td/test-erasure-eio/6/new.json 2026-02-24T17:23:49.919 INFO:tasks.workunit.client.0.trial066.stdout:6 2026-02-24T17:23:49.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-24T17:23:49.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCl3p1pqF/JKhAAvsuC4Y+aV57MXy3D8TgBng== --osd-uuid 2be4f51b-5ca1-4993-8eb9-37a1ec6e6ed7 2026-02-24T17:23:49.945 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:49.952+0000 7f29b4506a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:49.946 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:49.952+0000 7f29b4506a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:49.947 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:49.956+0000 7f29b4506a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:49.948 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:49.956+0000 7f29b4506a40 -1 bdev(0x55597f132400 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-24T17:23:49.948 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:49.956+0000 7f29b4506a40 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-24T17:23:50.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-24T17:23:50.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:23:50.612 INFO:tasks.workunit.client.0.trial066.stdout:adding osd6 key to auth repository 2026-02-24T17:23:50.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-24T17:23:50.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:23:50.898 INFO:tasks.workunit.client.0.trial066.stdout:start osd.6 2026-02-24T17:23:50.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-24T17:23:50.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:23:50.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:23:50.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:23:50.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:23:50.921 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:50.928+0000 7fd4d0fbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:50.923 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:50.932+0000 7fd4d0fbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:50.924 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:50.932+0000 7fd4d0fbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:51.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:23:51.304 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:51.312+0000 7fd4d0fbfa40 -1 Falling back to public interface 2026-02-24T17:23:51.327 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:23:51.905 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:23:51.912+0000 7fd4d0fbfa40 -1 osd.6 0 log_to_monitors true 2026-02-24T17:23:52.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:23:52.328 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:23:52.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:23:52.329 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:23:52.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:23:52.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-24T17:23:52.541 INFO:tasks.workunit.client.0.trial066.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/357142413,v1:127.0.0.1:6851/357142413] [v2:127.0.0.1:6852/357142413,v1:127.0.0.1:6853/357142413] exists,up 2be4f51b-5ca1-4993-8eb9-37a1ec6e6ed7 2026-02-24T17:23:52.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:23:52.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:23:52.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:23:52.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:23:52.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:23:52.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:23:52.542 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:23:52.543 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:52.543 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:52.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:23:52.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:23:52.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:23:52.597 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:23:52.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:23:52.605 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:23:28.328+0000 7fac39ecba40 0 load: jerasure load: lrc 2026-02-24T17:23:52.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-24T17:23:52.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-24T17:23:52.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:23:52.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:23:52.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-24T17:23:52.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:23:52.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-24T17:23:52.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:23:52.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-24T17:23:52.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:23:52.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:23:53.254 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:23:53.263 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:23:54.265 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:23:54.265 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:23:54.265 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:23:54.265 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:23:54.266 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:23:54.266 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:23:54.266 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:23:54.267 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:23:54.267 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:23:54.267 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:23:54.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:23:54.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:23:54.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:23:54.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:23:54.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:23:54.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:23:54.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:54.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:23:54.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803783 2026-02-24T17:23:54.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803783 2026-02-24T17:23:54.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783' 2026-02-24T17:23:54.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:54.676 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:23:54.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607559 2026-02-24T17:23:54.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607559 2026-02-24T17:23:54.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559' 2026-02-24T17:23:54.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:54.756 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:23:54.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444038 2026-02-24T17:23:54.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444038 2026-02-24T17:23:54.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038' 2026-02-24T17:23:54.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:54.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:23:54.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182406 2026-02-24T17:23:54.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182406 2026-02-24T17:23:54.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-107374182406' 2026-02-24T17:23:54.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:54.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:23:54.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=146028888068 2026-02-24T17:23:54.995 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 146028888068 2026-02-24T17:23:54.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-107374182406 4-146028888068' 2026-02-24T17:23:54.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:54.996 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:23:55.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593732 2026-02-24T17:23:55.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593732 2026-02-24T17:23:55.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-107374182406 4-146028888068 5-184683593732' 2026-02-24T17:23:55.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:23:55.074 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:23:55.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397507 2026-02-24T17:23:55.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397507 2026-02-24T17:23:55.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-73014444038 3-107374182406 4-146028888068 5-184683593732 6-210453397507' 2026-02-24T17:23:55.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:55.155 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803783 2026-02-24T17:23:55.155 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:55.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:23:55.156 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803783 2026-02-24T17:23:55.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:55.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803783 2026-02-24T17:23:55.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-24T17:23:55.158 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803783 2026-02-24T17:23:55.158 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:23:55.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-24T17:23:55.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:23:56.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:23:56.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:23:56.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803783 2026-02-24T17:23:56.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:56.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607559 2026-02-24T17:23:56.603 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:56.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:23:56.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607559 2026-02-24T17:23:56.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:56.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607559 2026-02-24T17:23:56.606 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607559 2026-02-24T17:23:56.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-24T17:23:56.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:23:56.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-24T17:23:56.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:56.826 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444038 2026-02-24T17:23:56.827 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:56.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:23:56.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444038 2026-02-24T17:23:56.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:56.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444038 2026-02-24T17:23:56.830 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 73014444038 2026-02-24T17:23:56.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444038' 2026-02-24T17:23:56.830 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:23:57.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444038 -lt 73014444038 2026-02-24T17:23:57.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:57.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182406 2026-02-24T17:23:57.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:57.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:23:57.059 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182406 2026-02-24T17:23:57.059 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:57.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182406 2026-02-24T17:23:57.061 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182406 2026-02-24T17:23:57.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-24T17:23:57.061 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:23:57.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-24T17:23:57.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:57.277 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-146028888068 2026-02-24T17:23:57.277 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:57.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:23:57.279 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-146028888068 2026-02-24T17:23:57.279 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:57.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=146028888068 2026-02-24T17:23:57.280 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 146028888068 2026-02-24T17:23:57.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-24T17:23:57.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:23:57.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888069 -lt 146028888068 2026-02-24T17:23:57.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:57.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-184683593732 2026-02-24T17:23:57.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:57.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:23:57.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-184683593732 2026-02-24T17:23:57.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:57.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593732 2026-02-24T17:23:57.508 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 184683593732 2026-02-24T17:23:57.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-24T17:23:57.509 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:23:57.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-24T17:23:57.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:23:57.728 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 6-210453397507 2026-02-24T17:23:57.728 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:23:57.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=6 2026-02-24T17:23:57.730 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 6-210453397507 2026-02-24T17:23:57.730 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:23:57.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397507 2026-02-24T17:23:57.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-24T17:23:57.732 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.6 seq 210453397507 2026-02-24T17:23:57.732 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-24T17:23:57.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-24T17:23:57.939 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:23:57.939 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:23:57.940 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:23:58.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:23:58.212 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:23:58.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:23:58.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:23:58.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:23:58.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:23:58.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:23:58.213 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:23:58.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:23:58.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:23:58.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:23:58.428 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:23:58.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:23:58.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:23:58.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:23:58.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:23:58.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:23:58.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:23:58.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:23:58.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-24T17:23:58.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-24T17:23:58.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=myobject 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure myobject 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:23:58.738 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:23:58.956 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:23:58.957 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-24T17:23:58.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:23:58.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-24T17:23:58.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:23:58.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-24T17:23:58.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:23:58.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:23:58.959 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:23:58.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-24T17:23:58.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:23:58.959 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:23:58.960 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:23:58.960 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:58.960 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:58.960 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:23:58.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:23:59.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:23:59.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:23:59.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:23:59.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-24T17:23:59.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:23:59.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:23:59.022 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:23:59.022 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:23:59.022 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:23:59.023 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:23:59.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:23:59.023 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-24T17:23:59.081 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-24T17:23:59.082 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:23:59.082 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-24T17:23:59.082 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-24T17:23:59.083 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:23:59.307 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-24T17:23:59.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-24T17:23:59.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:23:59.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:23:59.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:23:59.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:23:59.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:23:59.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:23:59.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-24T17:23:59.828 INFO:tasks.workunit.client.0.trial066.stderr:osd.6 is already down. 2026-02-24T17:23:59.837 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-24T17:24:00.082 INFO:tasks.workunit.client.0.trial066.stderr:osd.6 is already out. 2026-02-24T17:24:00.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-24T17:24:00.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:24:00.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:24:00.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:24:00.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:24:00.095 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:24:00.095 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:24:00.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:24:00.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:24:00.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:24:00.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:24:00.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:24:00.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:24:00.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:24:00.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:24:00.198 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:24:00.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:24:00.415 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:00.415 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:24:00.415 INFO:tasks.workunit.client.0.trial066.stderr:3 2026-02-24T17:24:00.415 INFO:tasks.workunit.client.0.trial066.stderr:4 2026-02-24T17:24:00.415 INFO:tasks.workunit.client.0.trial066.stderr:5 2026-02-24T17:24:00.415 INFO:tasks.workunit.client.0.trial066.stderr:6' 2026-02-24T17:24:00.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:24:00.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:00.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:24:00.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803787 2026-02-24T17:24:00.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803787 2026-02-24T17:24:00.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787' 2026-02-24T17:24:00.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:00.494 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:24:00.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607562 2026-02-24T17:24:00.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607562 2026-02-24T17:24:00.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562' 2026-02-24T17:24:00.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:00.574 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:24:00.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444041 2026-02-24T17:24:00.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444041 2026-02-24T17:24:00.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444041' 2026-02-24T17:24:00.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:00.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:24:00.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182409 2026-02-24T17:24:00.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182409 2026-02-24T17:24:00.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444041 3-107374182409' 2026-02-24T17:24:00.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:00.735 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.4 flush_pg_stats 2026-02-24T17:24:00.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=146028888072 2026-02-24T17:24:00.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 146028888072 2026-02-24T17:24:00.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444041 3-107374182409 4-146028888072' 2026-02-24T17:24:00.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:00.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.5 flush_pg_stats 2026-02-24T17:24:00.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=184683593735 2026-02-24T17:24:00.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 184683593735 2026-02-24T17:24:00.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-73014444041 3-107374182409 4-146028888072 5-184683593735' 2026-02-24T17:24:00.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:00.892 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.6 flush_pg_stats 2026-02-24T17:24:00.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq= 2026-02-24T17:24:00.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: true 2026-02-24T17:24:00.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z '' 2026-02-24T17:24:00.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: flush_pg_stats: continue 2026-02-24T17:24:00.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:00.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803787 2026-02-24T17:24:00.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:00.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:24:00.956 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803787 2026-02-24T17:24:00.956 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:00.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803787 2026-02-24T17:24:00.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-24T17:24:00.957 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803787 2026-02-24T17:24:00.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:01.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803787 2026-02-24T17:24:01.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:02.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:24:02.179 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:02.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-24T17:24:02.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:02.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607562 2026-02-24T17:24:02.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:02.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:24:02.398 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607562 2026-02-24T17:24:02.399 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:02.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607562 2026-02-24T17:24:02.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607562' 2026-02-24T17:24:02.400 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607562 2026-02-24T17:24:02.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:02.608 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607562 -lt 51539607562 2026-02-24T17:24:02.609 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:02.609 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444041 2026-02-24T17:24:02.609 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:02.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:24:02.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444041 2026-02-24T17:24:02.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:02.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444041 2026-02-24T17:24:02.614 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 73014444041 2026-02-24T17:24:02.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444041' 2026-02-24T17:24:02.614 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:02.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444041 -lt 73014444041 2026-02-24T17:24:02.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:02.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182409 2026-02-24T17:24:02.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:02.838 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:24:02.839 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182409 2026-02-24T17:24:02.839 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:02.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182409 2026-02-24T17:24:02.841 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182409 2026-02-24T17:24:02.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182409' 2026-02-24T17:24:02.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:24:03.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182410 -lt 107374182409 2026-02-24T17:24:03.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:03.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 4-146028888072 2026-02-24T17:24:03.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:03.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=4 2026-02-24T17:24:03.067 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 4-146028888072 2026-02-24T17:24:03.067 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:03.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=146028888072 2026-02-24T17:24:03.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.4 seq 146028888072' 2026-02-24T17:24:03.068 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.4 seq 146028888072 2026-02-24T17:24:03.068 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-24T17:24:03.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 146028888072 -lt 146028888072 2026-02-24T17:24:03.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:03.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 5-184683593735 2026-02-24T17:24:03.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:03.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=5 2026-02-24T17:24:03.296 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 5-184683593735 2026-02-24T17:24:03.296 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:03.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=184683593735 2026-02-24T17:24:03.298 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.5 seq 184683593735 2026-02-24T17:24:03.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.5 seq 184683593735' 2026-02-24T17:24:03.298 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-24T17:24:03.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 184683593735 -lt 184683593735 2026-02-24T17:24:03.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:24:03.524 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:03.524 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:03.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:24:03.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:24:03.812 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:24:03.812 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:24:03.812 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:24:03.812 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:24:03.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:24:03.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:24:04.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:24:04.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:24:04.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:04.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:04.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:24:04.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:24:04.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:24:04.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-24T17:24:04.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:24:04.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:24:04.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-24T17:24:04.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:24:04.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:24:04.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-24T17:24:04.329 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:24:04.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:24:04.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:24:04.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:24:04.332 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:24:04.601 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:24:04.612 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:24:04.905 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:24:04.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:24:04.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:24:04.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:24:04.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:24:04.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:24:04.916 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:24:04.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:24:04.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:24:04.916 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:24:05.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:24:05.230 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:24:05.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:24:05.232 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:24:05.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:24:05.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:24:05.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:24:05.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:24:05.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:24:05.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:24:05.236 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:24:05.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:24:05.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:24:05.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:24:05.252 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:24:05.253 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:05.253 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:05.253 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:24:05.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:24:05.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:24:05.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:24:05.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:24:05.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:24:05.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:24:05.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:24:05.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:24:05.258 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:24:05.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:24:05.259 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:24:05.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:24:05.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:24:05.260 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:24:05.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:24:05.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:24:05.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:24:05.262 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:24:05.263 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:24:05.264 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:24:05.264 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:24:05.266 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:24:05.266 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:05.266 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:05.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:24:05.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:24:05.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:24:05.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:24:05.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:24:05.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:05.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:24:05.271 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:24:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:24:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:24:05.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:24:05.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:05.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:05.294 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:05.294 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:05.295 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:05.295 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:05.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:24:05.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:24:05.322 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:24:05.322 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:24:05.322 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:24:05.322 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:24:05.323 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:24:05.323 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:24:05.323 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:24:05.323 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:24:05.324 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:24:05.324 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:05.324 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:05.324 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:24:05.324 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:24:05.324 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:24:05.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:24:05.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:24:05.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:24:05.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:24:05.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:24:05.382 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:24:05.382 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:24:05.382 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:24:05.382 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:24:05.383 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:05.383 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:05.383 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:24:05.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:24:05.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:24:05.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:24:05.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:24:05.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:24:05.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:24:05.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:24:05.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:24:05.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:24:05.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:24:05.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:05.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:05.559 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:05.559 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:05.559 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:05.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:05.560 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:24:05.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:24:05.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:24:05.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:24:05.682 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:24:05.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:24:06.693 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:24:06.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:24:06.742 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:24:06.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:24:06.750 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:24:05.320+0000 7fc714623ec0 0 load: jerasure load: lrc 2026-02-24T17:24:06.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-24T17:24:06.750 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-24T17:24:06.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-24T17:24:06.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-24T17:24:06.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:24:06.751 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-24T17:24:06.752 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-24T17:24:06.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:24:06.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:24:06.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:06.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:06.754 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:24:06.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:24:06.756 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:24:06.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=726f722b-fae9-4c36-b17d-5a45e47df509 2026-02-24T17:24:06.757 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 726f722b-fae9-4c36-b17d-5a45e47df509 2026-02-24T17:24:06.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 726f722b-fae9-4c36-b17d-5a45e47df509' 2026-02-24T17:24:06.757 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:24:06.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC23p1pqHhSLhAA6IAriXl46SmA7Ut6SL8Oqw== 2026-02-24T17:24:06.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC23p1pqHhSLhAA6IAriXl46SmA7Ut6SL8Oqw=="}' 2026-02-24T17:24:06.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 726f722b-fae9-4c36-b17d-5a45e47df509 -i td/test-erasure-eio/0/new.json 2026-02-24T17:24:06.880 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:24:06.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:24:06.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC23p1pqHhSLhAA6IAriXl46SmA7Ut6SL8Oqw== --osd-uuid 726f722b-fae9-4c36-b17d-5a45e47df509 2026-02-24T17:24:06.905 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:06.912+0000 7f8c9a5bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:06.907 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:06.916+0000 7f8c9a5bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:06.908 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:06.916+0000 7f8c9a5bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:06.908 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:06.916+0000 7f8c9a5bba40 -1 bdev(0x55ed01a36400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:24:06.908 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:06.916+0000 7f8c9a5bba40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:24:07.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:24:07.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:24:07.579 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:24:07.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:24:07.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:24:07.715 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:24:07.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:24:07.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:24:07.716 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:24:07.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:24:07.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:07.753 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:07.760+0000 7f6263d5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:07.755 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:07.760+0000 7f6263d5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:07.756 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:07.764+0000 7f6263d5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:07.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:24:07.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:07.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:24:07.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:07.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:07.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:07.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:07.849 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:24:07.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:07.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:24:07.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:08.133 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:08.140+0000 7f6263d5ba40 -1 Falling back to public interface 2026-02-24T17:24:08.745 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:08.752+0000 7f6263d5ba40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:24:08.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:08.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:08.959 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:24:08.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:08.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:08.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:24:09.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:10.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:10.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:10.179 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:24:10.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:24:10.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:10.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:24:10.396 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1536418835,v1:127.0.0.1:6803/1536418835] [v2:127.0.0.1:6804/1536418835,v1:127.0.0.1:6805/1536418835] exists,up 726f722b-fae9-4c36-b17d-5a45e47df509 2026-02-24T17:24:10.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:10.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:10.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:10.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:10.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:24:10.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:24:10.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:24:10.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:24:10.403 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:24:10.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 d65f646b-9b67-4e8f-ae93-fc1181fdac2f' 2026-02-24T17:24:10.404 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:24:10.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC63p1pIiscGRAAV3Xw+nK212O1mKiOGzpAiA== 2026-02-24T17:24:10.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC63p1pIiscGRAAV3Xw+nK212O1mKiOGzpAiA=="}' 2026-02-24T17:24:10.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d65f646b-9b67-4e8f-ae93-fc1181fdac2f -i td/test-erasure-eio/1/new.json 2026-02-24T17:24:10.628 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:24:10.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:24:10.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC63p1pIiscGRAAV3Xw+nK212O1mKiOGzpAiA== --osd-uuid d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:24:10.654 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:10.660+0000 7efd96cdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:10.655 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:10.664+0000 7efd96cdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:10.656 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:10.664+0000 7efd96cdaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:10.657 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:10.664+0000 7efd96cdaa40 -1 bdev(0x56420f2aa400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:24:10.657 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:10.664+0000 7efd96cdaa40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:24:11.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:24:11.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:24:11.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:24:11.355 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:24:11.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:24:11.644 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:24:11.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:24:11.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:11.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:24:11.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:24:11.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:24:11.668 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:11.676+0000 7f2bf23dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:11.669 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:11.676+0000 7f2bf23dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:11.670 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:11.676+0000 7f2bf23dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:11.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:24:11.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:11.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:24:11.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:11.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:11.863 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:11.864 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:24:11.864 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:11.864 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:11.864 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:24:12.077 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:12.089 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:12.096+0000 7f2bf23dda40 -1 Falling back to public interface 2026-02-24T17:24:12.753 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:12.760+0000 7f2bf23dda40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:24:13.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:13.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:13.079 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:24:13.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:13.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:13.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:24:13.295 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:14.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:14.297 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:14.298 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:24:14.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:24:14.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:14.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:24:14.509 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1084636799,v1:127.0.0.1:6811/1084636799] [v2:127.0.0.1:6812/1084636799,v1:127.0.0.1:6813/1084636799] exists,up d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:24:14.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:24:14.514 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:14.515 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:24:14.516 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:24:14.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bf9b1288-ac2d-4e49-bcc6-46c5a4374254 2026-02-24T17:24:14.517 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 bf9b1288-ac2d-4e49-bcc6-46c5a4374254 2026-02-24T17:24:14.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 bf9b1288-ac2d-4e49-bcc6-46c5a4374254' 2026-02-24T17:24:14.518 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:24:14.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+3p1p+8vqHxAA08UQG881NUdl8U8K2yA9dQ== 2026-02-24T17:24:14.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+3p1p+8vqHxAA08UQG881NUdl8U8K2yA9dQ=="}' 2026-02-24T17:24:14.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bf9b1288-ac2d-4e49-bcc6-46c5a4374254 -i td/test-erasure-eio/2/new.json 2026-02-24T17:24:14.749 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:24:14.759 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:24:14.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+3p1p+8vqHxAA08UQG881NUdl8U8K2yA9dQ== --osd-uuid bf9b1288-ac2d-4e49-bcc6-46c5a4374254 2026-02-24T17:24:14.785 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:14.792+0000 7f7363709a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:14.787 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:14.792+0000 7f7363709a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:14.788 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:14.796+0000 7f7363709a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:14.789 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:14.796+0000 7f7363709a40 -1 bdev(0x5585cfbb0400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:24:14.789 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:14.796+0000 7f7363709a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:24:15.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:24:15.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:24:15.444 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:24:15.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:24:15.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:24:15.732 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:24:15.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:24:15.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:15.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:24:15.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:24:15.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:24:15.756 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:15.764+0000 7fabff136a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:15.758 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:15.764+0000 7fabff136a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:15.759 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:15.764+0000 7fabff136a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:15.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:15.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:24:16.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:16.237 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:16.244+0000 7fabff136a40 -1 Falling back to public interface 2026-02-24T17:24:16.866 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:16.872+0000 7fabff136a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:24:17.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:17.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:17.155 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:24:17.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:17.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:17.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:24:17.370 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:18.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:18.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:18.373 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:24:18.373 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:24:18.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:18.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:24:18.587 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/70601672,v1:127.0.0.1:6819/70601672] [v2:127.0.0.1:6820/70601672,v1:127.0.0.1:6821/70601672] exists,up bf9b1288-ac2d-4e49-bcc6-46c5a4374254 2026-02-24T17:24:18.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:24:18.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:18.590 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:18.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:24:18.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 cd2bae0b-5d17-4f25-8406-f2fc882fc7ce' 2026-02-24T17:24:18.593 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:24:18.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDC3p1pzGGaJBAAj7CYGGfAx6iOy62aNXtSlQ== 2026-02-24T17:24:18.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDC3p1pzGGaJBAAj7CYGGfAx6iOy62aNXtSlQ=="}' 2026-02-24T17:24:18.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cd2bae0b-5d17-4f25-8406-f2fc882fc7ce -i td/test-erasure-eio/3/new.json 2026-02-24T17:24:18.824 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:24:18.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:24:18.835 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDC3p1pzGGaJBAAj7CYGGfAx6iOy62aNXtSlQ== --osd-uuid cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:24:18.852 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:18.860+0000 7f8e83999a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:18.853 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:18.860+0000 7f8e83999a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:18.854 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:18.860+0000 7f8e83999a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:18.855 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:18.860+0000 7f8e83999a40 -1 bdev(0x55c29e412400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:24:18.855 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:18.860+0000 7f8e83999a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:24:19.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:24:19.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:24:19.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:24:19.531 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:24:19.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:24:19.831 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:24:19.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:24:19.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:19.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:24:19.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:24:19.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:24:19.848 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:19.856+0000 7faf6dd76a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:19.849 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:19.856+0000 7faf6dd76a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:19.850 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:19.856+0000 7faf6dd76a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:20.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:24:20.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:20.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:24:20.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:20.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:20.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:20.047 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:24:20.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:20.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:20.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:20.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:20.309 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:20.316+0000 7faf6dd76a40 -1 Falling back to public interface 2026-02-24T17:24:20.928 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:20.936+0000 7faf6dd76a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:24:21.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:21.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:21.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:21.259 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:24:21.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:21.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:21.480 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:22.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:22.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:22.482 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:24:22.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:24:22.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:22.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:22.703 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3033647719,v1:127.0.0.1:6827/3033647719] [v2:127.0.0.1:6828/3033647719,v1:127.0.0.1:6829/3033647719] exists,up cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:24:22.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:22.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:22.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:22.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:24:22.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:24:22.704 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:24:22.704 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:24:22.704 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:22.704 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:22.704 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:24:22.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:24:22.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:24:22.760 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:24:22.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:24:22.768 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:24:08.144+0000 7f6263d5ba40 0 load: jerasure load: lrc 2026-02-24T17:24:22.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-24T17:24:22.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-24T17:24:22.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:24:22.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:24:22.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-24T17:24:22.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:24:22.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-24T17:24:22.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:24:22.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-24T17:24:23.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:24:23.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:24:23.413 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:24:23.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:24:24.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:24:24.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:24:24.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:24:24.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:24:24.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:24:24.426 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:24:24.426 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:24:24.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:24:24.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:24:24.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:24:24.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:24:24.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:24:24.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:24:24.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:24:24.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:24:24.522 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:24:24.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:24:24.736 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:24.736 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:24:24.736 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:24:24.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:24:24.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:24.736 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:24:24.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-24T17:24:24.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-24T17:24:24.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-24T17:24:24.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:24.819 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:24:24.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607557 2026-02-24T17:24:24.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607557 2026-02-24T17:24:24.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607557' 2026-02-24T17:24:24.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:24.898 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:24:24.976 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444036 2026-02-24T17:24:24.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444036 2026-02-24T17:24:24.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-73014444036' 2026-02-24T17:24:24.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:24.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:24:25.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182403 2026-02-24T17:24:25.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182403 2026-02-24T17:24:25.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607557 2-73014444036 3-107374182403' 2026-02-24T17:24:25.056 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:25.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-24T17:24:25.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:25.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:24:25.057 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-24T17:24:25.057 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:25.058 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-24T17:24:25.058 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803781 2026-02-24T17:24:25.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-24T17:24:25.059 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:25.284 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-24T17:24:25.284 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:26.285 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:24:26.285 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:26.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803781 2026-02-24T17:24:26.503 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:26.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607557 2026-02-24T17:24:26.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:26.505 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:24:26.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607557 2026-02-24T17:24:26.506 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:26.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607557 2026-02-24T17:24:26.507 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607557 2026-02-24T17:24:26.508 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607557' 2026-02-24T17:24:26.508 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:26.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607557 -lt 51539607557 2026-02-24T17:24:26.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:26.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444036 2026-02-24T17:24:26.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:26.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:24:26.720 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444036 2026-02-24T17:24:26.720 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:26.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444036 2026-02-24T17:24:26.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-24T17:24:26.721 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 73014444036 2026-02-24T17:24:26.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:26.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-24T17:24:26.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:26.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182403 2026-02-24T17:24:26.943 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:26.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:24:26.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182403 2026-02-24T17:24:26.945 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:26.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182403 2026-02-24T17:24:26.946 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182403 2026-02-24T17:24:26.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-24T17:24:26.947 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:24:27.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-24T17:24:27.173 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:24:27.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:27.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:27.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:24:27.460 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:24:27.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:24:27.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:24:27.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:24:27.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:24:27.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:24:27.461 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:24:27.679 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:24:27.679 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:24:27.679 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:27.679 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:27.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:24:27.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:24:27.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:24:27.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-24T17:24:27.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-73729-0-10 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-73729-0-10 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:24:27.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-73729-0-10 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:24:27.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-73729-0-10 td/test-erasure-eio/ORIGINAL 2026-02-24T17:24:27.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-73729-0-10 td/test-erasure-eio 0 10 set 2026-02-24T17:24:27.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-0-10 2026-02-24T17:24:27.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:24:27.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:24:27.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:24:27.987 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-0-10 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-0-10 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-0-10 2026-02-24T17:24:27.988 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:24:28.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:24:28.199 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:28.199 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:24:28.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:24:28.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:24:28.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:24:28.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-24T17:24:28.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:24:28.510 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:24:28.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:24:28.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-24T17:24:28.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:10 bytes copied, 3.3412e-05 s, 299 kB/s 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-24T17:24:28.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:24:28.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-24T17:24:28.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:29.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:24:29.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:24:29.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T17:24:29.142 INFO:tasks.workunit.client.0.trial066.stderr:start osd.3 2026-02-24T17:24:29.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:29.142 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-24T17:24:29.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T17:24:29.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:24:29.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:24:29.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:24:29.164 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:29.171+0000 7f202f85ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:29.166 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:29.171+0000 7f202f85ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:29.166 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:29.171+0000 7f202f85ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:29.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T17:24:29.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:29.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:24:29.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:29.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:29.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:29.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:29.355 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:24:29.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:29.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:29.521 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:29.527+0000 7f202f85ba40 -1 Falling back to public interface 2026-02-24T17:24:29.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:30.161 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:30.167+0000 7f202f85ba40 -1 osd.3 35 log_to_monitors true 2026-02-24T17:24:30.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:30.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:30.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:30.574 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:30.574 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:30.575 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:osd.3 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [25,35) [v2:127.0.0.1:6826/669988225,v1:127.0.0.1:6827/669988225] [v2:127.0.0.1:6828/669988225,v1:127.0.0.1:6829/669988225] exists,up cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:24:30.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:24:30.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:24:30.793 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:24:30.793 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:24:30.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:24:30.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:24:30.794 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:24:30.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:24:30.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:24:30.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:24:30.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:24:30.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:24:30.901 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:24:31.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:24:31.113 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:31.113 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:24:31.113 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:24:31.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:24:31.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:31.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:24:31.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803785 2026-02-24T17:24:31.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803785 2026-02-24T17:24:31.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785' 2026-02-24T17:24:31.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:31.196 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:24:31.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607560 2026-02-24T17:24:31.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607560 2026-02-24T17:24:31.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-51539607560' 2026-02-24T17:24:31.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:31.276 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:24:31.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444039 2026-02-24T17:24:31.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444039 2026-02-24T17:24:31.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-51539607560 2-73014444039' 2026-02-24T17:24:31.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:31.359 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:24:31.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724547 2026-02-24T17:24:31.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724547 2026-02-24T17:24:31.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-51539607560 2-73014444039 3-167503724547' 2026-02-24T17:24:31.437 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:31.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803785 2026-02-24T17:24:31.438 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:31.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:24:31.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803785 2026-02-24T17:24:31.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:31.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803785 2026-02-24T17:24:31.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-24T17:24:31.440 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803785 2026-02-24T17:24:31.441 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:31.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803785 2026-02-24T17:24:31.657 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:32.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:24:32.658 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:32.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-24T17:24:32.871 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:32.871 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607560 2026-02-24T17:24:32.871 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:32.872 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:24:32.873 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607560 2026-02-24T17:24:32.873 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:32.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607560 2026-02-24T17:24:32.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607560' 2026-02-24T17:24:32.875 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 51539607560 2026-02-24T17:24:32.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:33.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607560 -lt 51539607560 2026-02-24T17:24:33.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:33.092 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444039 2026-02-24T17:24:33.092 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:33.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:24:33.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444039 2026-02-24T17:24:33.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444039 2026-02-24T17:24:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444039' 2026-02-24T17:24:33.095 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444039 2026-02-24T17:24:33.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:33.315 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-24T17:24:33.316 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:33.316 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-167503724547 2026-02-24T17:24:33.316 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:33.317 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:24:33.318 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-167503724547 2026-02-24T17:24:33.318 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:33.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724547 2026-02-24T17:24:33.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 167503724547' 2026-02-24T17:24:33.320 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 167503724547 2026-02-24T17:24:33.320 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:24:33.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-24T17:24:33.531 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:24:33.531 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:33.532 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:33.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:24:33.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:24:33.820 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:24:33.820 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:24:33.820 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:24:33.820 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:24:33.820 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:24:33.820 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:24:34.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:24:34.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:24:34.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:34.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:34.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:24:34.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:24:34.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:24:34.299 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:24:34.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:24:34.616 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:24:34.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-0-10 2026-02-24T17:24:34.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:24:34.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:24:34.629 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-0-10 2026-02-24T17:24:34.629 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:24:34.629 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:24:34.629 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-0-10 td/test-erasure-eio/COPY 2026-02-24T17:24:34.654 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:24:34.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:24:34.657 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-73729-0-10 td/test-erasure-eio 1 10 set 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-0-10 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-0-10 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:24:34.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-0-10 2026-02-24T17:24:34.660 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-0-10 2026-02-24T17:24:34.660 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-24T17:24:34.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:24:35.184 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:24:35.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:24:35.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-24T17:24:35.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-24T17:24:35.197 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:10 bytes copied, 3.0416e-05 s, 329 kB/s 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:24:35.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:24:35.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:24:35.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:24:35.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:24:35.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:24:35.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:24:35.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:24:35.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:24:35.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:24:35.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:35.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:24:35.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:24:35.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:24:35.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:24:35.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:24:35.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:24:35.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:35.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:24:35.830 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:24:35.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:35.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:24:35.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:24:35.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:24:35.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:24:35.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:24:35.853 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:35.859+0000 7f230731ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:35.854 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:35.859+0000 7f230731ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:35.855 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:35.863+0000 7f230731ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:36.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:36.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:24:36.267 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:36.317 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:36.323+0000 7f230731ba40 -1 Falling back to public interface 2026-02-24T17:24:36.942 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:36.947+0000 7f230731ba40 -1 osd.1 43 log_to_monitors true 2026-02-24T17:24:37.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:37.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:37.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:37.269 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:37.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:37.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:24:37.482 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [12,43) [v2:127.0.0.1:6810/2122549113,v1:127.0.0.1:6811/2122549113] [v2:127.0.0.1:6812/2122549113,v1:127.0.0.1:6813/2122549113] exists,up d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:24:37.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:37.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:37.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:37.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:24:37.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:24:37.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:24:37.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:24:37.483 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:24:37.484 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:24:37.484 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:24:37.484 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:24:37.484 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:24:37.484 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:24:37.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:24:37.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:24:37.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:24:37.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:24:37.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:24:37.585 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:24:37.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:24:37.804 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:37.804 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:24:37.804 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:24:37.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:24:37.804 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:37.805 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:24:37.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803788 2026-02-24T17:24:37.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803788 2026-02-24T17:24:37.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788' 2026-02-24T17:24:37.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:37.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:24:37.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397507 2026-02-24T17:24:37.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397507 2026-02-24T17:24:37.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-210453397507' 2026-02-24T17:24:37.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:37.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:24:38.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444043 2026-02-24T17:24:38.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444043 2026-02-24T17:24:38.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-73014444043' 2026-02-24T17:24:38.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:38.042 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:24:38.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724550 2026-02-24T17:24:38.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724550 2026-02-24T17:24:38.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-73014444043 3-167503724550' 2026-02-24T17:24:38.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:38.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803788 2026-02-24T17:24:38.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:38.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:24:38.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803788 2026-02-24T17:24:38.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:38.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803788 2026-02-24T17:24:38.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-24T17:24:38.125 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803788 2026-02-24T17:24:38.125 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:38.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-24T17:24:38.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:39.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:24:39.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:39.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-24T17:24:39.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:40.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:24:40.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:40.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-24T17:24:40.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:40.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-210453397507 2026-02-24T17:24:40.774 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:40.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:24:40.776 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-210453397507 2026-02-24T17:24:40.776 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:40.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397507 2026-02-24T17:24:40.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 210453397507' 2026-02-24T17:24:40.777 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 210453397507 2026-02-24T17:24:40.778 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:40.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-24T17:24:40.996 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:40.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444043 2026-02-24T17:24:40.997 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:40.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:24:40.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444043 2026-02-24T17:24:40.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:41.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444043 2026-02-24T17:24:41.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444043' 2026-02-24T17:24:41.000 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444043 2026-02-24T17:24:41.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:41.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444043 -lt 73014444043 2026-02-24T17:24:41.229 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:41.230 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-167503724550 2026-02-24T17:24:41.230 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:41.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:24:41.232 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-167503724550 2026-02-24T17:24:41.232 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:41.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724550 2026-02-24T17:24:41.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 167503724550' 2026-02-24T17:24:41.233 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 167503724550 2026-02-24T17:24:41.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:24:41.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-24T17:24:41.457 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:24:41.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:41.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:41.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:24:41.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:24:41.744 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:24:41.744 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:24:41.744 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:24:41.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:24:41.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:24:41.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:24:41.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:24:41.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:24:41.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:41.966 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:42.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:24:42.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:24:42.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:24:42.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:24:42.258 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:24:42.580 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-0-10 fail 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-0-10 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-24T17:24:42.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-73729-0-10 td/test-erasure-eio/COPY 2026-02-24T17:24:42.613 INFO:tasks.workunit.client.0.trial066.stderr:error getting pool-jerasure/obj-size-73729-0-10: (5) Input/output error 2026-02-24T17:24:42.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-24T17:24:42.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:24:42.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-24T17:24:42.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-24T17:24:42.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-24T17:24:42.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-24T17:24:42.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-24T17:24:42.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-73729-0-0 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-73729-0-0 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-73729-0-0 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:24:42.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-73729-0-0 td/test-erasure-eio/ORIGINAL 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-73729-0-0 td/test-erasure-eio 0 0 set 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-0-0 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:24:42.638 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:24:42.639 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-0-0 2026-02-24T17:24:42.639 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:24:42.639 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-0-0 2026-02-24T17:24:42.639 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-0-0 2026-02-24T17:24:42.639 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:24:42.847 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:24:42.847 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:42.847 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:24:42.847 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:24:42.847 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:24:42.847 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:24:42.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-24T17:24:42.848 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:24:43.140 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:24:43.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:24:43.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-24T17:24:43.152 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-24T17:24:43.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:43.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:24:43.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:24:43.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:24:43.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:24:43.464 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:24:43.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:43.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:24:43.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:24:43.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-24T17:24:43.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:24:43.465 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-24T17:24:43.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:43.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:43.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T17:24:43.742 INFO:tasks.workunit.client.0.trial066.stderr:start osd.3 2026-02-24T17:24:43.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:43.743 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-24T17:24:43.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T17:24:43.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:24:43.744 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:24:43.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:24:43.760 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:43.767+0000 7f1f34517a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:43.761 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:43.767+0000 7f1f34517a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:43.762 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:43.767+0000 7f1f34517a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:43.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T17:24:43.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:43.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:44.129 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:44.135+0000 7f1f34517a40 -1 Falling back to public interface 2026-02-24T17:24:44.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:44.768 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:44.775+0000 7f1f34517a40 -1 osd.3 54 log_to_monitors true 2026-02-24T17:24:45.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:45.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:45.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:45.179 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:45.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:45.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:45.399 INFO:tasks.workunit.client.0.trial066.stderr:osd.3 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [39,54) [v2:127.0.0.1:6826/1622285162,v1:127.0.0.1:6827/1622285162] [v2:127.0.0.1:6828/1622285162,v1:127.0.0.1:6829/1622285162] exists,up cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:24:45.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:24:45.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:24:45.401 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:24:45.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:24:45.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:24:45.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:24:45.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:24:45.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:24:45.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:24:45.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:24:45.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:24:45.512 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:24:45.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:24:45.725 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:45.725 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:24:45.725 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:24:45.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:24:45.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:45.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:24:45.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803792 2026-02-24T17:24:45.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803792 2026-02-24T17:24:45.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792' 2026-02-24T17:24:45.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:45.807 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:24:45.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=210453397510 2026-02-24T17:24:45.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 210453397510 2026-02-24T17:24:45.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-210453397510' 2026-02-24T17:24:45.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:45.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:24:45.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444046 2026-02-24T17:24:45.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444046 2026-02-24T17:24:45.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-210453397510 2-73014444046' 2026-02-24T17:24:45.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:45.971 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:24:46.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=253403070467 2026-02-24T17:24:46.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 253403070467 2026-02-24T17:24:46.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-210453397510 2-73014444046 3-253403070467' 2026-02-24T17:24:46.049 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:46.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803792 2026-02-24T17:24:46.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:46.050 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:24:46.051 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803792 2026-02-24T17:24:46.051 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:46.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803792 2026-02-24T17:24:46.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-24T17:24:46.052 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803792 2026-02-24T17:24:46.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:46.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-24T17:24:46.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:46.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-210453397510 2026-02-24T17:24:46.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:46.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:24:46.281 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-210453397510 2026-02-24T17:24:46.281 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:46.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=210453397510 2026-02-24T17:24:46.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-24T17:24:46.283 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 210453397510 2026-02-24T17:24:46.283 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:46.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397508 -lt 210453397510 2026-02-24T17:24:46.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:47.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:24:47.504 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:47.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397508 -lt 210453397510 2026-02-24T17:24:47.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:48.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:24:48.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:48.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 210453397511 -lt 210453397510 2026-02-24T17:24:48.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:48.933 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444046 2026-02-24T17:24:48.933 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:48.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:24:48.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444046 2026-02-24T17:24:48.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:48.937 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444046 2026-02-24T17:24:48.937 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444046' 2026-02-24T17:24:48.937 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444046 2026-02-24T17:24:48.938 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:49.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444046 -lt 73014444046 2026-02-24T17:24:49.156 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:49.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-253403070467 2026-02-24T17:24:49.157 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:49.158 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:24:49.159 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-253403070467 2026-02-24T17:24:49.159 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:49.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=253403070467 2026-02-24T17:24:49.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 253403070467' 2026-02-24T17:24:49.161 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 253403070467 2026-02-24T17:24:49.161 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:24:49.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-24T17:24:49.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:24:49.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:49.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:49.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:24:49.673 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:24:49.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:24:49.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:24:49.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:24:49.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:24:49.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:24:49.674 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:24:49.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:24:49.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:24:49.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:49.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:50.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:24:50.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:24:50.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:24:50.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:24:50.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:24:50.497 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:24:50.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-0-0 2026-02-24T17:24:50.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:24:50.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:24:50.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-0-0 2026-02-24T17:24:50.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:24:50.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:24:50.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-0-0 td/test-erasure-eio/COPY 2026-02-24T17:24:50.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:24:50.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:24:50.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-24T17:24:50.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-24T17:24:50.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-73729-0-0 td/test-erasure-eio 1 0 set 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-0-0 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-0-0 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:24:50.543 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-0-0 2026-02-24T17:24:50.544 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-0-0 2026-02-24T17:24:50.544 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:24:50.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:24:50.764 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:50.764 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:24:50.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:24:50.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:24:50.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:24:50.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-24T17:24:50.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:24:51.054 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:24:51.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:24:51.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-24T17:24:51.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-24T17:24:51.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:24:51.068 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:24:51.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:24:51.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:24:51.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:24:51.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:24:51.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:51.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:24:51.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:24:51.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:24:51.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:24:51.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:24:51.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:24:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:24:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:24:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:24:51.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:51.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:24:51.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:51.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:51.663 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:51.663 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:51.663 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:51.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:24:51.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:24:51.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:24:51.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:24:51.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:24:51.681 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:51.687+0000 7f5c098eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:51.682 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:51.687+0000 7f5c098eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:51.683 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:51.687+0000 7f5c098eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:51.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:24:51.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:51.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:24:51.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:51.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:51.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:51.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:51.885 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:24:51.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:51.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:24:52.085 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:52.091+0000 7f5c098eea40 -1 Falling back to public interface 2026-02-24T17:24:52.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:24:52.814 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:52.819+0000 7f5c098eea40 -1 osd.1 63 log_to_monitors true 2026-02-24T17:24:53.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:24:53.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:53.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:24:53.098 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:53.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:53.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:24:53.312 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [49,63) [v2:127.0.0.1:6810/2697131774,v1:127.0.0.1:6811/2697131774] [v2:127.0.0.1:6812/2697131774,v1:127.0.0.1:6813/2697131774] exists,up d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:24:53.313 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:24:53.314 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:24:53.314 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:24:53.314 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:24:53.314 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:24:53.314 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:24:53.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:24:53.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:24:53.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:24:53.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:24:53.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:24:53.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:24:53.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:24:53.630 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:53.630 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:24:53.630 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:24:53.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:24:53.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:53.631 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:24:53.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803795 2026-02-24T17:24:53.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803795 2026-02-24T17:24:53.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795' 2026-02-24T17:24:53.710 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:53.711 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:24:53.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776131 2026-02-24T17:24:53.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776131 2026-02-24T17:24:53.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-292057776131' 2026-02-24T17:24:53.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:53.795 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:24:53.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444050 2026-02-24T17:24:53.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444050 2026-02-24T17:24:53.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-292057776131 2-73014444050' 2026-02-24T17:24:53.873 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:24:53.873 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:24:53.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=253403070470 2026-02-24T17:24:53.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 253403070470 2026-02-24T17:24:53.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-292057776131 2-73014444050 3-253403070470' 2026-02-24T17:24:53.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:53.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803795 2026-02-24T17:24:53.955 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:53.956 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:24:53.957 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803795 2026-02-24T17:24:53.957 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:53.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803795 2026-02-24T17:24:53.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-24T17:24:53.958 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803795 2026-02-24T17:24:53.959 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:24:54.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803795 -lt 25769803795 2026-02-24T17:24:54.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:54.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-292057776131 2026-02-24T17:24:54.181 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:54.182 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:24:54.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-292057776131 2026-02-24T17:24:54.183 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:54.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776131 2026-02-24T17:24:54.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-24T17:24:54.184 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 292057776131 2026-02-24T17:24:54.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:24:54.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-24T17:24:54.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:54.405 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444050 2026-02-24T17:24:54.405 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:54.406 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:24:54.407 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444050 2026-02-24T17:24:54.407 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:54.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444050 2026-02-24T17:24:54.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444050' 2026-02-24T17:24:54.408 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444050 2026-02-24T17:24:54.409 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:54.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444048 -lt 73014444050 2026-02-24T17:24:54.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:55.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:24:55.628 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:55.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444048 -lt 73014444050 2026-02-24T17:24:55.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:24:56.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:24:56.845 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:24:57.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444050 -lt 73014444050 2026-02-24T17:24:57.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:24:57.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-253403070470 2026-02-24T17:24:57.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:24:57.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:24:57.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-253403070470 2026-02-24T17:24:57.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:24:57.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=253403070470 2026-02-24T17:24:57.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 253403070470' 2026-02-24T17:24:57.066 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 253403070470 2026-02-24T17:24:57.067 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:24:57.279 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 253403070471 -lt 253403070470 2026-02-24T17:24:57.279 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:24:57.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:57.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:57.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:24:57.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:24:57.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:24:57.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:24:57.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:24:57.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:24:57.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:24:57.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:24:57.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:24:57.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:24:57.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:24:57.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:24:58.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:24:58.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:24:58.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:24:58.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:24:58.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:24:58.367 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:24:58.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-24T17:24:58.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-0-0 fail 2026-02-24T17:24:58.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:24:58.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:24:58.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-0-0 2026-02-24T17:24:58.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-24T17:24:58.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-24T17:24:58.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-73729-0-0 td/test-erasure-eio/COPY 2026-02-24T17:24:58.405 INFO:tasks.workunit.client.0.trial066.stderr:error getting pool-jerasure/obj-size-73729-0-0: (5) Input/output error 2026-02-24T17:24:58.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-24T17:24:58.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:24:58.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-24T17:24:58.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-24T17:24:58.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-24T17:24:58.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-24T17:24:58.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-24T17:24:58.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-73729-0-256 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-73729-0-256 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-73729-0-256 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:58.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:24:58.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:58.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:24:58.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:24:58.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:24:58.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-73729-0-256 td/test-erasure-eio/ORIGINAL 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-73729-0-256 td/test-erasure-eio 0 256 add 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-0-256 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:24:58.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-24T17:24:58.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:24:58.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-24T17:24:58.436 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:24:58.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-0-256 2026-02-24T17:24:58.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:24:58.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-0-256 2026-02-24T17:24:58.437 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-0-256 2026-02-24T17:24:58.437 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:24:58.654 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:24:58.654 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:24:58.654 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:24:58.655 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:24:58.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:24:58.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:24:58.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-24T17:24:58.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:24:58.940 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:24:58.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-24T17:24:58.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:58.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:24:58.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:24:58.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:24:58.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:24:59.259 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:24:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:24:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:24:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-24T17:24:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:24:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-24T17:24:59.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:24:59.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:24:59.424 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:24:59.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:24:59.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T17:24:59.426 INFO:tasks.workunit.client.0.trial066.stderr:start osd.3 2026-02-24T17:24:59.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:24:59.426 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-24T17:24:59.427 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T17:24:59.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:24:59.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:24:59.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:24:59.444 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:59.451+0000 7f0b5db83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:59.445 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:59.451+0000 7f0b5db83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:59.446 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:59.451+0000 7f0b5db83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:24:59.646 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:24:59.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:24:59.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:24:59.857 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:24:59.863+0000 7f0b5db83a40 -1 Falling back to public interface 2026-02-24T17:24:59.864 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:00.501 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:00.507+0000 7f0b5db83a40 -1 osd.3 71 log_to_monitors true 2026-02-24T17:25:00.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:00.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:00.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:00.867 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:00.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:00.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:25:01.081 INFO:tasks.workunit.client.0.trial066.stderr:osd.3 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,71) [v2:127.0.0.1:6826/3257189658,v1:127.0.0.1:6827/3257189658] [v2:127.0.0.1:6828/3257189658,v1:127.0.0.1:6829/3257189658] exists,up cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:25:01.082 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:25:01.083 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:25:01.083 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:25:01.083 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:25:01.083 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:25:01.083 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:25:01.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:25:01.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:25:01.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:25:01.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:25:01.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:25:01.191 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:25:01.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:25:01.413 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:01.413 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:25:01.413 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:25:01.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:25:01.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:01.413 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:25:01.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803799 2026-02-24T17:25:01.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803799 2026-02-24T17:25:01.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799' 2026-02-24T17:25:01.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:01.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:25:01.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776134 2026-02-24T17:25:01.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776134 2026-02-24T17:25:01.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-292057776134' 2026-02-24T17:25:01.573 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:01.573 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:25:01.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444053 2026-02-24T17:25:01.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444053 2026-02-24T17:25:01.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-292057776134 2-73014444053' 2026-02-24T17:25:01.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:01.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:25:01.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=330712481795 2026-02-24T17:25:01.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 330712481795 2026-02-24T17:25:01.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-292057776134 2-73014444053 3-330712481795' 2026-02-24T17:25:01.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:01.732 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803799 2026-02-24T17:25:01.732 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:01.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:25:01.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803799 2026-02-24T17:25:01.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:01.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803799 2026-02-24T17:25:01.736 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803799' 2026-02-24T17:25:01.736 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803799 2026-02-24T17:25:01.736 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:01.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803799 -lt 25769803799 2026-02-24T17:25:01.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:01.947 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-292057776134 2026-02-24T17:25:01.947 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:01.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:25:01.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-292057776134 2026-02-24T17:25:01.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776134 2026-02-24T17:25:01.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 292057776134' 2026-02-24T17:25:01.951 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 292057776134 2026-02-24T17:25:01.951 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:25:02.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-24T17:25:02.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:02.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444053 2026-02-24T17:25:02.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:02.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:25:02.171 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444053 2026-02-24T17:25:02.171 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:02.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444053 2026-02-24T17:25:02.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444053' 2026-02-24T17:25:02.172 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444053 2026-02-24T17:25:02.173 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:02.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444053 -lt 73014444053 2026-02-24T17:25:02.386 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:02.387 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-330712481795 2026-02-24T17:25:02.387 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:02.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:25:02.389 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-330712481795 2026-02-24T17:25:02.389 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:02.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=330712481795 2026-02-24T17:25:02.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 330712481795' 2026-02-24T17:25:02.390 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 330712481795 2026-02-24T17:25:02.391 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:25:02.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-24T17:25:02.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:25:02.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:02.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:02.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:25:02.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:02.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:02.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:02.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:02.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:02.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:02.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:03.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-24T17:25:03.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:03.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:03.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:03.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-24T17:25:03.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' -1 2026-02-24T17:25:03.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-24T17:25:03.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=4 2026-02-24T17:25:03.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T17:25:03.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-24T17:25:03.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T17:25:03.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:03.512 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:03.512 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:03.512 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:03.512 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:03.513 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:03.513 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:03.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-24T17:25:03.728 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:03.729 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:03.729 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:04.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-24T17:25:04.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-24T17:25:04.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T17:25:04.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T17:25:04.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T17:25:04.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T17:25:04.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T17:25:04.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T17:25:04.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T17:25:04.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T17:25:04.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T17:25:04.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 13 )) 2026-02-24T17:25:04.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T17:25:04.313 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-24T17:25:04.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T17:25:04.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:04.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:04.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:04.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:04.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:04.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:04.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:04.728 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-24T17:25:04.729 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:04.729 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:04.729 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:05.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 5 2026-02-24T17:25:05.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 4 '!=' 4 2026-02-24T17:25:05.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-24T17:25:05.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-24T17:25:05.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-24T17:25:05.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-24T17:25:05.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-24T17:25:05.016 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-24T17:25:05.016 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-24T17:25:05.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-24T17:25:05.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-24T17:25:05.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 13 )) 2026-02-24T17:25:05.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-24T17:25:05.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-24T17:25:05.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-24T17:25:05.705 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:05.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:05.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:05.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:05.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:05.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:05.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:05.919 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:25:05.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:05.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:05.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:06.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:25:06.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:25:06.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:25:06.206 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-24T17:25:06.206 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:25:06.206 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:25:06.206 INFO:tasks.workunit.client.0.trial066.stderr:256 bytes copied, 2.2793e-05 s, 11.2 MB/s 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:25:06.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:25:06.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:25:06.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:25:06.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:06.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:25:06.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:25:06.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-24T17:25:06.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:25:06.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-24T17:25:06.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:06.809 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-24T17:25:06.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:06.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:25:06.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-24T17:25:06.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:25:06.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:25:06.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:06.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:06.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:06.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:06.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:25:06.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:25:06.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:06.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:25:06.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:06.812 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:25:06.812 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:06.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:06.813 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:06.813 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:06.813 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:06.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:06.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:06.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:06.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:06.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:06.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:06.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:06.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:06.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:06.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:25:06.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:25:06.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:25:06.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-24T17:25:06.815 INFO:tasks.workunit.client.0.trial066.stderr:start osd.3 2026-02-24T17:25:06.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:06.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-24T17:25:06.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-24T17:25:06.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:25:06.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:25:06.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:25:06.824 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:06.831+0000 7f848678ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:06.825 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:06.831+0000 7f848678ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:06.826 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:06.831+0000 7f848678ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:07.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:07.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:25:07.213 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:07.219+0000 7f848678ea40 -1 Falling back to public interface 2026-02-24T17:25:07.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:07.906 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:07.911+0000 7f848678ea40 -1 osd.3 79 log_to_monitors true 2026-02-24T17:25:08.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:08.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:08.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:08.245 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:08.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:08.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:25:08.458 INFO:tasks.workunit.client.0.trial066.stderr:osd.3 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [77,79) [v2:127.0.0.1:6826/3068039424,v1:127.0.0.1:6827/3068039424] [v2:127.0.0.1:6828/3068039424,v1:127.0.0.1:6829/3068039424] exists,up cd2bae0b-5d17-4f25-8406-f2fc882fc7ce 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:25:08.459 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:25:08.460 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:25:08.460 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:25:08.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:25:08.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:25:08.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:25:08.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:25:08.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:25:08.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:25:08.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:25:08.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:25:08.569 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:25:08.794 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:25:08.794 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:08.795 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:25:08.795 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:25:08.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:25:08.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:08.795 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:25:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803802 2026-02-24T17:25:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803802 2026-02-24T17:25:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802' 2026-02-24T17:25:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:08.877 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:25:08.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=292057776138 2026-02-24T17:25:08.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 292057776138 2026-02-24T17:25:08.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-292057776138' 2026-02-24T17:25:08.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:08.953 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:25:09.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444057 2026-02-24T17:25:09.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444057 2026-02-24T17:25:09.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-292057776138 2-73014444057' 2026-02-24T17:25:09.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:09.035 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:25:09.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285571 2026-02-24T17:25:09.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285571 2026-02-24T17:25:09.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-292057776138 2-73014444057 3-356482285571' 2026-02-24T17:25:09.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:09.114 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803802 2026-02-24T17:25:09.114 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:09.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:25:09.115 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803802 2026-02-24T17:25:09.116 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:09.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803802 2026-02-24T17:25:09.117 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-24T17:25:09.117 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803802 2026-02-24T17:25:09.117 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:09.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803800 -lt 25769803802 2026-02-24T17:25:09.330 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:25:10.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:25:10.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:10.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-24T17:25:10.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:10.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-292057776138 2026-02-24T17:25:10.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:10.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:25:10.556 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-292057776138 2026-02-24T17:25:10.557 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:10.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=292057776138 2026-02-24T17:25:10.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 292057776138' 2026-02-24T17:25:10.558 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 292057776138 2026-02-24T17:25:10.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:25:10.783 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 292057776138 -lt 292057776138 2026-02-24T17:25:10.783 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:10.784 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444057 2026-02-24T17:25:10.784 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:10.785 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:25:10.786 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444057 2026-02-24T17:25:10.786 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:10.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444057 2026-02-24T17:25:10.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444057' 2026-02-24T17:25:10.787 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444057 2026-02-24T17:25:10.788 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:11.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444057 -lt 73014444057 2026-02-24T17:25:11.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:11.007 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-356482285571 2026-02-24T17:25:11.007 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:11.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:25:11.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-356482285571 2026-02-24T17:25:11.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:11.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285571 2026-02-24T17:25:11.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 356482285571' 2026-02-24T17:25:11.012 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 356482285571 2026-02-24T17:25:11.012 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:25:11.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-24T17:25:11.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:25:11.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:11.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:11.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:25:11.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:11.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:11.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:11.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:11.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:11.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:11.526 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:11.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:25:11.744 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:11.744 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:11.744 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:25:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:25:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:25:12.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:25:12.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:25:12.340 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:25:12.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-0-256 2026-02-24T17:25:12.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:25:12.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:25:12.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-0-256 2026-02-24T17:25:12.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:25:12.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:25:12.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-0-256 td/test-erasure-eio/COPY 2026-02-24T17:25:12.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:25:12.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:25:12.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-24T17:25:12.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-24T17:25:12.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-73729-0-256 td/test-erasure-eio 1 256 add 2026-02-24T17:25:12.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-0-256 2026-02-24T17:25:12.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:25:12.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:25:12.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-0-256 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:25:12.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-0-256 2026-02-24T17:25:12.384 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-0-256 2026-02-24T17:25:12.384 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:25:12.597 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:25:12.598 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:12.598 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:25:12.598 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:25:12.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:25:12.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:25:12.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-24T17:25:12.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:25:12.909 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:25:12.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:25:12.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:25:12.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:25:12.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:25:12.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:25:12.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:25:12.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:25:12.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:25:12.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:25:13.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:25:13.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:13.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:25:13.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:25:13.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:25:13.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:25:13.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:25:13.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:13.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:13.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:25:13.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:25:13.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:25:13.402 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:25:13.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:13.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:25:13.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:25:13.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:25:13.404 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:25:13.405 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:25:13.420 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:13.427+0000 7fbef13dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:13.422 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:13.427+0000 7fbef13dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:13.423 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:13.427+0000 7fbef13dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:13.621 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:25:13.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:13.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:13.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:13.885 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:13.891+0000 7fbef13dba40 -1 Falling back to public interface 2026-02-24T17:25:14.476 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:14.479+0000 7fbef13dba40 -1 osd.1 86 log_to_monitors true 2026-02-24T17:25:14.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:14.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:14.839 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:14.839 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:14.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:14.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 92 up_thru 92 down_at 89 last_clean_interval [68,86) [v2:127.0.0.1:6810/117439637,v1:127.0.0.1:6811/117439637] [v2:127.0.0.1:6812/117439637,v1:127.0.0.1:6813/117439637] exists,up d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:25:15.051 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:25:15.052 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:25:15.052 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:25:15.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:25:15.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:25:15.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:25:15.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:25:15.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:25:15.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:25:15.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:25:15.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:25:15.161 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:25:15.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:25:15.371 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:15.371 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:25:15.371 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:25:15.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:25:15.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:15.372 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:25:15.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803806 2026-02-24T17:25:15.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803806 2026-02-24T17:25:15.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803806' 2026-02-24T17:25:15.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:15.454 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:25:15.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=395136991235 2026-02-24T17:25:15.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 395136991235 2026-02-24T17:25:15.532 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803806 1-395136991235' 2026-02-24T17:25:15.532 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:15.532 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:25:15.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444060 2026-02-24T17:25:15.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444060 2026-02-24T17:25:15.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803806 1-395136991235 2-73014444060' 2026-02-24T17:25:15.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:15.611 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:25:15.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285574 2026-02-24T17:25:15.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285574 2026-02-24T17:25:15.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803806 1-395136991235 2-73014444060 3-356482285574' 2026-02-24T17:25:15.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:15.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803806 2026-02-24T17:25:15.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:15.692 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:25:15.693 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803806 2026-02-24T17:25:15.693 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:15.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803806 2026-02-24T17:25:15.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803806' 2026-02-24T17:25:15.695 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803806 2026-02-24T17:25:15.695 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:15.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803806 -lt 25769803806 2026-02-24T17:25:15.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:15.911 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-395136991235 2026-02-24T17:25:15.911 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:15.913 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:25:15.913 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-395136991235 2026-02-24T17:25:15.914 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:15.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=395136991235 2026-02-24T17:25:15.915 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 395136991235' 2026-02-24T17:25:15.915 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 395136991235 2026-02-24T17:25:15.915 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:25:16.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-24T17:25:16.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:16.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444060 2026-02-24T17:25:16.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:16.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:25:16.131 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444060 2026-02-24T17:25:16.131 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:16.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444060 2026-02-24T17:25:16.133 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444060' 2026-02-24T17:25:16.133 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444060 2026-02-24T17:25:16.133 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:16.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444060 -lt 73014444060 2026-02-24T17:25:16.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:16.352 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-356482285574 2026-02-24T17:25:16.352 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:16.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:25:16.353 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-356482285574 2026-02-24T17:25:16.354 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:16.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285574 2026-02-24T17:25:16.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 356482285574' 2026-02-24T17:25:16.356 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 356482285574 2026-02-24T17:25:16.356 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:25:16.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285574 -lt 356482285574 2026-02-24T17:25:16.584 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:25:16.584 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:16.584 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:16.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:25:16.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:16.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:16.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:16.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:16.875 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:16.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:16.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:17.086 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:25:17.087 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:17.088 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:17.088 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:17.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:25:17.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:25:17.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:25:17.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-24T17:25:17.367 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:25:17.367 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:25:17.367 INFO:tasks.workunit.client.0.trial066.stderr:256 bytes copied, 3.7951e-05 s, 6.7 MB/s 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:25:17.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:25:17.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:25:17.369 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:25:17.369 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:25:17.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:25:17.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:25:17.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:25:17.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:25:17.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:17.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:25:17.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:25:17.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:25:17.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:25:17.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:25:17.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:25:17.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:17.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:25:17.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:17.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:17.965 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:17.965 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:17.965 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:17.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:25:17.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:25:17.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:25:17.967 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:25:17.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:17.967 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:25:17.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:25:17.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:25:17.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:25:17.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:25:17.987 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:17.991+0000 7f69c4c51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:17.988 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:17.995+0000 7f69c4c51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:17.989 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:17.995+0000 7f69c4c51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:18.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:25:18.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:18.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:25:18.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:18.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:18.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:18.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:18.186 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:25:18.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:18.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:18.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:18.441 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:18.447+0000 7f69c4c51a40 -1 Falling back to public interface 2026-02-24T17:25:19.041 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:19.047+0000 7f69c4c51a40 -1 osd.1 93 log_to_monitors true 2026-02-24T17:25:19.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:19.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:19.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:19.396 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:19.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:19.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:19.613 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 97 up_thru 97 down_at 94 last_clean_interval [92,93) [v2:127.0.0.1:6810/1475859612,v1:127.0.0.1:6811/1475859612] [v2:127.0.0.1:6812/1475859612,v1:127.0.0.1:6813/1475859612] exists,up d65f646b-9b67-4e8f-ae93-fc1181fdac2f 2026-02-24T17:25:19.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:19.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:19.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:19.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:25:19.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:25:19.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:25:19.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:25:19.614 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:25:19.615 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:25:19.615 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:25:19.615 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:25:19.615 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:25:19.615 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:25:19.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:25:19.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:25:19.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:25:19.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:25:19.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:25:19.714 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:25:19.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:25:19.930 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:19.930 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:25:19.930 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:25:19.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:25:19.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:19.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:25:20.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803809 2026-02-24T17:25:20.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803809 2026-02-24T17:25:20.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809' 2026-02-24T17:25:20.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:20.012 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:25:20.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=416611827715 2026-02-24T17:25:20.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 416611827715 2026-02-24T17:25:20.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-416611827715' 2026-02-24T17:25:20.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:20.092 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:25:20.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444063 2026-02-24T17:25:20.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444063 2026-02-24T17:25:20.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-416611827715 2-73014444063' 2026-02-24T17:25:20.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:20.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:25:20.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=356482285577 2026-02-24T17:25:20.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 356482285577 2026-02-24T17:25:20.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803809 1-416611827715 2-73014444063 3-356482285577' 2026-02-24T17:25:20.248 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:20.249 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803809 2026-02-24T17:25:20.249 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:20.249 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:25:20.250 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803809 2026-02-24T17:25:20.250 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:20.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803809 2026-02-24T17:25:20.252 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803809' 2026-02-24T17:25:20.252 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803809 2026-02-24T17:25:20.252 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:20.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803806 -lt 25769803809 2026-02-24T17:25:20.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:25:21.471 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:25:21.472 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:21.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803806 -lt 25769803809 2026-02-24T17:25:21.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:25:22.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:25:22.681 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:22.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803809 -lt 25769803809 2026-02-24T17:25:22.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:22.900 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-416611827715 2026-02-24T17:25:22.900 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:22.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:25:22.902 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-416611827715 2026-02-24T17:25:22.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:22.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=416611827715 2026-02-24T17:25:22.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 416611827715' 2026-02-24T17:25:22.904 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 416611827715 2026-02-24T17:25:22.904 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:25:23.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-24T17:25:23.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:23.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444063 2026-02-24T17:25:23.121 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:23.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:25:23.122 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444063 2026-02-24T17:25:23.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:23.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444063 2026-02-24T17:25:23.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444063' 2026-02-24T17:25:23.124 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 73014444063 2026-02-24T17:25:23.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:23.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444063 -lt 73014444063 2026-02-24T17:25:23.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:23.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-356482285577 2026-02-24T17:25:23.336 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:23.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:25:23.338 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-356482285577 2026-02-24T17:25:23.338 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:23.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=356482285577 2026-02-24T17:25:23.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 356482285577' 2026-02-24T17:25:23.340 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 356482285577 2026-02-24T17:25:23.340 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:25:23.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 356482285577 -lt 356482285577 2026-02-24T17:25:23.557 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:25:23.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:23.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:23.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:25:23.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:23.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:23.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:23.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:23.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:23.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:23.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:24.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:25:24.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:24.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:24.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:24.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:25:24.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:25:24.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:25:24.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:25:24.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:25:24.660 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-0-256 fail 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-0-256 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-24T17:25:24.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-73729-0-256 td/test-erasure-eio/COPY 2026-02-24T17:25:24.694 INFO:tasks.workunit.client.0.trial066.stderr:error getting pool-jerasure/obj-size-73729-0-256: (5) Input/output error 2026-02-24T17:25:24.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-24T17:25:24.697 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:25:24.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:25:24.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:25:24.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:25:24.920 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:25:24.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:25:25.225 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:25:25.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:25:25.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:25:25.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:25:25.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:25:25.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:25:25.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:25:25.552 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:25:25.552 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:25:25.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:25:25.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:25:25.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:25:25.554 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:25:25.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:25:25.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:25:25.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:25:25.582 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:25:25.582 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:25.582 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:25.583 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:25:25.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:25:25.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:25:25.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:25:25.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:25:25.588 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:25:25.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:25:25.589 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:25:25.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:25:25.590 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:25:25.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:25:25.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:25:25.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:25:25.592 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:25:25.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:25:25.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:25:25.594 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:25:25.595 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:25:25.596 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:25:25.596 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:25.596 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:25.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:25:25.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:25:25.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:25:25.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:25:25.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:25:25.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:25.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:25.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:25:25.600 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:25:25.602 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:25:25.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:25:25.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:25.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:25.625 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:25.626 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:25.626 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:25.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:25.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:25:25.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:25:25.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:25:25.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:25:25.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:25:25.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:25:25.652 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:25:25.652 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:25:25.652 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:25:25.652 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:25:25.652 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:25:25.652 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:25.652 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:25.653 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:25:25.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:25:25.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:25:25.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:25:25.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:25:25.725 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:25:25.726 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:25:25.726 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:25:25.727 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:25:25.727 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:25:25.727 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:25:25.727 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:25:25.727 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:25.727 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:25.727 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:25:25.728 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:25:25.728 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:25:25.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:25:25.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:25:25.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:25:25.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:25:25.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:25:25.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:25:25.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:25:25.901 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:25:25.902 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:25.902 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:25.902 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:25.902 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:25.902 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:25.902 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:25.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:25:25.903 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:25:25.919 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:25:25.919 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:25:26.026 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:25:26.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:25:27.035 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:25:27.035 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:25:27.035 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:25:27.036 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:25:27.036 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:27.036 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:27.036 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:25:27.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:25:27.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:25:27.087 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:25:27.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:25:27.095 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:25:25.646+0000 7f15303c2ec0 0 load: jerasure load: lrc 2026-02-24T17:25:27.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-24T17:25:27.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-24T17:25:27.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-24T17:25:27.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-24T17:25:27.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:25:27.096 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-24T17:25:27.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:27.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:27.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:27.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:27.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:27.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:25:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f168b297-02ea-41aa-b8ea-190d40d7a587 2026-02-24T17:25:27.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f168b297-02ea-41aa-b8ea-190d40d7a587' 2026-02-24T17:25:27.103 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 f168b297-02ea-41aa-b8ea-190d40d7a587 2026-02-24T17:25:27.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:25:27.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAH351ptmlDBxAAZfNz1f2yQRX4gPqYQxlepw== 2026-02-24T17:25:27.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAH351ptmlDBxAAZfNz1f2yQRX4gPqYQxlepw=="}' 2026-02-24T17:25:27.113 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f168b297-02ea-41aa-b8ea-190d40d7a587 -i td/test-erasure-eio/0/new.json 2026-02-24T17:25:27.227 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:25:27.235 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:25:27.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAH351ptmlDBxAAZfNz1f2yQRX4gPqYQxlepw== --osd-uuid f168b297-02ea-41aa-b8ea-190d40d7a587 2026-02-24T17:25:27.253 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:27.258+0000 7f5af329ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:27.254 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:27.258+0000 7f5af329ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:27.255 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:27.258+0000 7f5af329ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:27.256 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:27.262+0000 7f5af329ba40 -1 bdev(0x55d32f69c400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:25:27.256 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:27.262+0000 7f5af329ba40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:25:27.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:25:27.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:25:27.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:25:27.965 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:25:27.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:25:28.114 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:25:28.114 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:25:28.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:28.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:25:28.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:25:28.115 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:25:28.129 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:28.134+0000 7f4941d9fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:28.130 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:28.134+0000 7f4941d9fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:28.131 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:28.138+0000 7f4941d9fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:28.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:25:28.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:28.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:25:28.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:28.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:28.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:28.224 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:25:28.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:28.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:28.225 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:25:28.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:28.545 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:28.550+0000 7f4941d9fa40 -1 Falling back to public interface 2026-02-24T17:25:29.117 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:29.122+0000 7f4941d9fa40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:25:29.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:29.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:29.451 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:25:29.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:29.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:29.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:25:29.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:30.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:30.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:30.666 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:25:30.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:25:30.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:30.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:25:30.876 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4029163180,v1:127.0.0.1:6803/4029163180] [v2:127.0.0.1:6804/4029163180,v1:127.0.0.1:6805/4029163180] exists,up f168b297-02ea-41aa-b8ea-190d40d7a587 2026-02-24T17:25:30.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:30.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:30.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:30.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:25:30.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:25:30.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:30.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:30.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:30.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:25:30.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:25:30.884 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:25:30.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:25:30.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 05cc947f-6450-4d33-abd9-18307b51f708' 2026-02-24T17:25:30.885 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:25:30.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:25:30.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAK351p8r/ONRAAUuL26YwkwIJluDCQUC15og== 2026-02-24T17:25:30.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAK351p8r/ONRAAUuL26YwkwIJluDCQUC15og=="}' 2026-02-24T17:25:30.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 05cc947f-6450-4d33-abd9-18307b51f708 -i td/test-erasure-eio/1/new.json 2026-02-24T17:25:31.109 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:25:31.118 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:25:31.119 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAK351p8r/ONRAAUuL26YwkwIJluDCQUC15og== --osd-uuid 05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:25:31.136 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:31.142+0000 7fdfaf9a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:31.138 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:31.142+0000 7fdfaf9a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:31.140 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:31.142+0000 7fdfaf9a3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:31.140 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:31.142+0000 7fdfaf9a3a40 -1 bdev(0x55abdc910400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:25:31.140 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:31.142+0000 7fdfaf9a3a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:25:31.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:25:31.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:25:31.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:25:31.812 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:25:31.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:25:32.094 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:25:32.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:25:32.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:32.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:25:32.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:25:32.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:25:32.119 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:32.122+0000 7fdb9e5c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:32.120 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:32.126+0000 7fdb9e5c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:32.122 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:32.126+0000 7fdb9e5c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:32.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:25:32.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:32.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:25:32.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:32.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:32.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:32.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:32.311 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:25:32.312 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:32.312 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:32.437 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:32.442+0000 7fdb9e5c0a40 -1 Falling back to public interface 2026-02-24T17:25:32.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:33.142 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:33.146+0000 7fdb9e5c0a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:25:33.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:33.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:33.523 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:25:33.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:33.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:33.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:33.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:34.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:34.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:34.747 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:25:34.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:25:34.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:34.748 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:34.960 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1850037318,v1:127.0.0.1:6811/1850037318] [v2:127.0.0.1:6812/1850037318,v1:127.0.0.1:6813/1850037318] exists,up 05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:25:34.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:34.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:34.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:34.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:25:34.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:25:34.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:34.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:34.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:25:34.966 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:25:34.967 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:25:34.968 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:25:34.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:25:34.968 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:25:34.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a' 2026-02-24T17:25:34.969 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:25:34.978 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAO351pr6XMOhAArdlJPr7vNRvXMcxRBZ0IBQ== 2026-02-24T17:25:34.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAO351pr6XMOhAArdlJPr7vNRvXMcxRBZ0IBQ=="}' 2026-02-24T17:25:34.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a -i td/test-erasure-eio/2/new.json 2026-02-24T17:25:35.192 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:25:35.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:25:35.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAO351pr6XMOhAArdlJPr7vNRvXMcxRBZ0IBQ== --osd-uuid 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:25:35.218 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:35.222+0000 7feca4c48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:35.220 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:35.226+0000 7feca4c48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:35.221 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:35.226+0000 7feca4c48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:35.221 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:35.226+0000 7feca4c48a40 -1 bdev(0x55741e7fe400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:25:35.221 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:35.226+0000 7feca4c48a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:25:35.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:25:35.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:25:35.891 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:25:35.892 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:25:35.892 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:25:36.176 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:25:36.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:25:36.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:36.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:25:36.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:25:36.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:25:36.201 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:36.206+0000 7f39045b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:36.202 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:36.206+0000 7f39045b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:36.204 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:36.210+0000 7f39045b9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:36.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:25:36.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:36.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:25:36.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:36.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:36.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:36.396 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:25:36.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:36.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:36.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:25:36.581 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:36.586+0000 7f39045b9a40 -1 Falling back to public interface 2026-02-24T17:25:36.608 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:37.202 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:37.206+0000 7f39045b9a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:25:37.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:37.610 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:37.610 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:25:37.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:37.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:37.611 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:25:37.823 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3491102237,v1:127.0.0.1:6819/3491102237] [v2:127.0.0.1:6820/3491102237,v1:127.0.0.1:6821/3491102237] exists,up 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:25:37.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:37.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:37.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:37.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:25:37.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:25:37.830 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:25:37.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3462da3a-3e5f-4df0-80a2-f03cda5af912 2026-02-24T17:25:37.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 3462da3a-3e5f-4df0-80a2-f03cda5af912' 2026-02-24T17:25:37.831 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 3462da3a-3e5f-4df0-80a2-f03cda5af912 2026-02-24T17:25:37.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:25:37.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAR351pquKmMhAAkx5CkZLayJZaIO+TrCcTKQ== 2026-02-24T17:25:37.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAR351pquKmMhAAkx5CkZLayJZaIO+TrCcTKQ=="}' 2026-02-24T17:25:37.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3462da3a-3e5f-4df0-80a2-f03cda5af912 -i td/test-erasure-eio/3/new.json 2026-02-24T17:25:38.055 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:25:38.065 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:25:38.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAR351pquKmMhAAkx5CkZLayJZaIO+TrCcTKQ== --osd-uuid 3462da3a-3e5f-4df0-80a2-f03cda5af912 2026-02-24T17:25:38.081 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:38.086+0000 7fd436a5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:38.082 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:38.086+0000 7fd436a5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:38.083 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:38.090+0000 7fd436a5ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:38.084 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:38.090+0000 7fd436a5ba40 -1 bdev(0x5592daca0400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:25:38.084 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:38.090+0000 7fd436a5ba40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:25:38.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:25:38.751 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:25:38.752 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:25:38.753 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:25:38.753 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:25:39.039 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:25:39.039 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:25:39.039 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:39.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:25:39.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:25:39.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:25:39.064 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:39.070+0000 7f5c45472a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:39.066 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:39.070+0000 7f5c45472a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:39.067 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:39.070+0000 7f5c45472a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:39.255 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:25:39.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:39.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:25:39.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:39.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:39.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:39.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:39.256 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:25:39.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:39.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:25:39.466 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:39.497 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:39.502+0000 7f5c45472a40 -1 Falling back to public interface 2026-02-24T17:25:40.102 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:40.106+0000 7f5c45472a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:25:40.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:40.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:40.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:40.468 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:25:40.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:40.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:25:40.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:41.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:41.676 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:41.676 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:25:41.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:25:41.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:41.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:25:41.886 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4175740587,v1:127.0.0.1:6827/4175740587] [v2:127.0.0.1:6828/4175740587,v1:127.0.0.1:6829/4175740587] exists,up 3462da3a-3e5f-4df0-80a2-f03cda5af912 2026-02-24T17:25:41.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:41.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:41.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:41.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:25:41.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:25:41.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:25:41.889 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:25:41.889 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:41.889 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:41.889 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:25:41.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:25:41.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:25:41.949 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:25:41.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:25:41.956 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:25:28.554+0000 7f4941d9fa40 0 load: jerasure load: lrc 2026-02-24T17:25:41.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-24T17:25:41.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-24T17:25:41.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:25:41.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:25:41.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-24T17:25:41.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:25:41.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-24T17:25:41.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:25:41.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-24T17:25:42.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:25:42.224 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:25:42.589 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:25:42.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:25:43.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:25:43.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:25:43.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:25:43.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:25:43.601 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:25:43.602 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:25:43.602 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:25:43.602 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:25:43.603 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:25:43.603 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:25:43.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:25:43.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:25:43.712 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:25:43.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:25:43.713 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:25:43.713 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:25:43.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:25:43.928 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:43.928 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:25:43.929 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:25:43.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:25:43.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:43.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:25:44.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-24T17:25:44.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-24T17:25:44.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-24T17:25:44.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:44.008 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:25:44.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607556 2026-02-24T17:25:44.091 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607556 2026-02-24T17:25:44.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-24T17:25:44.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:44.092 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:25:44.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411332 2026-02-24T17:25:44.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411332 2026-02-24T17:25:44.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411332' 2026-02-24T17:25:44.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:44.173 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:25:44.250 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182403 2026-02-24T17:25:44.250 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182403 2026-02-24T17:25:44.250 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411332 3-107374182403' 2026-02-24T17:25:44.250 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:44.251 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-24T17:25:44.251 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:44.251 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:25:44.252 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-24T17:25:44.252 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:44.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-24T17:25:44.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-24T17:25:44.254 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803781 2026-02-24T17:25:44.254 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:44.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-24T17:25:44.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:44.483 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607556 2026-02-24T17:25:44.483 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:44.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:25:44.485 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607556 2026-02-24T17:25:44.485 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:44.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607556 2026-02-24T17:25:44.486 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-24T17:25:44.487 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607556 2026-02-24T17:25:44.487 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:25:44.714 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-24T17:25:44.715 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:44.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411332 2026-02-24T17:25:44.715 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:44.717 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:25:44.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411332 2026-02-24T17:25:44.718 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:44.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411332 2026-02-24T17:25:44.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-24T17:25:44.719 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411332 2026-02-24T17:25:44.720 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:44.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411330 -lt 77309411332 2026-02-24T17:25:44.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:25:45.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:25:45.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:46.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411330 -lt 77309411332 2026-02-24T17:25:46.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:25:47.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:25:47.162 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:47.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-24T17:25:47.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:47.371 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182403 2026-02-24T17:25:47.372 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:47.373 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:25:47.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182403 2026-02-24T17:25:47.374 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:47.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182403 2026-02-24T17:25:47.375 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182403 2026-02-24T17:25:47.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-24T17:25:47.376 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:25:47.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-24T17:25:47.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:25:47.600 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:47.600 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:47.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:25:47.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:47.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:47.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:47.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:47.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:47.888 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:47.889 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:48.107 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:25:48.107 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:48.107 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:48.108 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:48.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:25:48.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-24T17:25:48.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-73729-1-10 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-73729-1-10 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-73729-1-10 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:25:48.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:25:48.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-73729-1-10 td/test-erasure-eio/ORIGINAL 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-73729-1-10 td/test-erasure-eio 1 10 set 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-1-10 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-24T17:25:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:25:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:25:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:25:48.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-1-10 2026-02-24T17:25:48.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:25:48.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-1-10 2026-02-24T17:25:48.416 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-1-10 2026-02-24T17:25:48.417 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-24T17:25:48.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:25:48.917 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:25:48.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:25:48.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-24T17:25:48.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-24T17:25:48.930 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:25:48.930 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:25:48.930 INFO:tasks.workunit.client.0.trial066.stderr:10 bytes copied, 3.1669e-05 s, 316 kB/s 2026-02-24T17:25:48.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:48.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:25:48.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:25:48.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:25:48.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:25:49.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:25:49.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:49.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:25:49.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:25:49.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:25:49.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:25:49.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:25:49.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:25:49.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:49.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:25:49.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:25:49.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:25:49.551 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:25:49.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:49.552 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:25:49.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:25:49.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:25:49.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:25:49.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:25:49.571 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:49.574+0000 7fe1bc34ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:49.572 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:49.578+0000 7fe1bc34ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:49.573 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:49.578+0000 7fe1bc34ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:49.768 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:25:49.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:49.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:49.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:50.017 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:50.022+0000 7fe1bc34ba40 -1 Falling back to public interface 2026-02-24T17:25:50.643 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:50.646+0000 7fe1bc34ba40 -1 osd.1 34 log_to_monitors true 2026-02-24T17:25:50.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:50.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:50.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:50.979 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:50.979 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:50.980 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:25:51.187 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [12,34) [v2:127.0.0.1:6810/306090676,v1:127.0.0.1:6811/306090676] [v2:127.0.0.1:6812/306090676,v1:127.0.0.1:6813/306090676] exists,up 05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:25:51.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:51.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:51.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:51.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:25:51.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:25:51.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:25:51.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:25:51.188 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:25:51.189 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:25:51.189 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:25:51.189 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:25:51.189 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:25:51.189 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:25:51.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:25:51.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:25:51.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:25:51.292 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:25:51.293 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:25:51.293 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:25:51.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:25:51.506 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:51.506 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:25:51.506 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:25:51.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:25:51.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:51.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:25:51.584 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803785 2026-02-24T17:25:51.584 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803785 2026-02-24T17:25:51.584 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785' 2026-02-24T17:25:51.584 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:51.585 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:25:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724547 2026-02-24T17:25:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724547 2026-02-24T17:25:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-167503724547' 2026-02-24T17:25:51.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:51.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:25:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411335 2026-02-24T17:25:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411335 2026-02-24T17:25:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-167503724547 2-77309411335' 2026-02-24T17:25:51.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:51.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:25:51.818 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182407 2026-02-24T17:25:51.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182407 2026-02-24T17:25:51.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803785 1-167503724547 2-77309411335 3-107374182407' 2026-02-24T17:25:51.819 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:51.819 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803785 2026-02-24T17:25:51.819 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:51.820 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:25:51.821 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803785 2026-02-24T17:25:51.821 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:51.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803785 2026-02-24T17:25:51.822 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-24T17:25:51.822 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803785 2026-02-24T17:25:51.823 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:52.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803785 2026-02-24T17:25:52.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:25:53.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:25:53.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:53.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-24T17:25:53.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:53.267 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-167503724547 2026-02-24T17:25:53.267 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:53.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:25:53.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-167503724547 2026-02-24T17:25:53.269 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:53.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724547 2026-02-24T17:25:53.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 167503724547' 2026-02-24T17:25:53.271 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 167503724547 2026-02-24T17:25:53.271 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:25:53.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-24T17:25:53.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:53.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411335 2026-02-24T17:25:53.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:53.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:25:53.498 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411335 2026-02-24T17:25:53.498 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411335 2026-02-24T17:25:53.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-24T17:25:53.499 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 77309411335 2026-02-24T17:25:53.499 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:25:53.718 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-24T17:25:53.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:53.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182407 2026-02-24T17:25:53.719 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:53.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:25:53.721 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182407 2026-02-24T17:25:53.721 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:53.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182407 2026-02-24T17:25:53.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182407' 2026-02-24T17:25:53.723 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182407 2026-02-24T17:25:53.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:25:53.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182407 -lt 107374182407 2026-02-24T17:25:53.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:25:53.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:53.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:54.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:25:54.234 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:25:54.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:25:54.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:25:54.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:25:54.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:25:54.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:25:54.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:25:54.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:25:54.450 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:25:54.451 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:25:54.451 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:25:54.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:25:54.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:25:54.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:25:54.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:25:54.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:25:55.053 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:25:55.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-1-10 2026-02-24T17:25:55.067 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:25:55.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:25:55.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-1-10 2026-02-24T17:25:55.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:25:55.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:25:55.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-1-10 td/test-erasure-eio/COPY 2026-02-24T17:25:55.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:25:55.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:25:55.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-24T17:25:55.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-24T17:25:55.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-73729-1-10 td/test-erasure-eio 2 10 set 2026-02-24T17:25:55.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-1-10 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-1-10 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:25:55.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-1-10 2026-02-24T17:25:55.101 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-1-10 2026-02-24T17:25:55.101 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:25:55.318 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:25:55.318 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:55.318 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:25:55.319 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:25:55.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:25:55.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:25:55.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-24T17:25:55.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:25:55.608 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:25:55.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:25:55.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-24T17:25:55.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-24T17:25:55.621 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:25:55.621 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:10 bytes copied, 3.3382e-05 s, 300 kB/s 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:25:55.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:25:55.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:25:55.623 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:25:55.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:25:55.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:25:55.623 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-24T17:25:55.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-73729-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:25:56.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-24T17:25:56.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:25:56.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:25:56.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:25:56.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:25:56.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:25:56.243 INFO:tasks.workunit.client.0.trial066.stderr:start osd.2 2026-02-24T17:25:56.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:25:56.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:25:56.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:25:56.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:25:56.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:25:56.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:25:56.263 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:56.266+0000 7f91fc997a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:56.264 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:56.270+0000 7f91fc997a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:56.265 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:56.270+0000 7f91fc997a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:25:56.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:25:56.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:25:56.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:25:56.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:25:56.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:25:56.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:56.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:25:56.462 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:25:56.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:56.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:25:56.645 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:56.650+0000 7f91fc997a40 -1 Falling back to public interface 2026-02-24T17:25:56.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:25:57.261 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:25:57.266+0000 7f91fc997a40 -1 osd.2 41 log_to_monitors true 2026-02-24T17:25:57.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:25:57.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:25:57.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:25:57.675 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:57.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:25:57.675 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:25:57.892 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 up in weight 1 up_from 48 up_thru 36 down_at 45 last_clean_interval [18,41) [v2:127.0.0.1:6818/1727626508,v1:127.0.0.1:6819/1727626508] [v2:127.0.0.1:6820/1727626508,v1:127.0.0.1:6821/1727626508] exists,up 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:25:57.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:25:57.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:25:57.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:25:57.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:25:57.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:25:57.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:25:57.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:25:57.894 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:25:57.894 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:25:57.894 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:25:57.894 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:25:57.894 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:25:57.895 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:25:58.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:25:58.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:25:58.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:25:58.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:25:58.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:25:58.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:25:58.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:25:58.222 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:25:58.222 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:25:58.222 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:25:58.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:25:58.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:58.222 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:25:58.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803788 2026-02-24T17:25:58.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803788 2026-02-24T17:25:58.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788' 2026-02-24T17:25:58.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:58.302 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:25:58.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724550 2026-02-24T17:25:58.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724550 2026-02-24T17:25:58.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-167503724550' 2026-02-24T17:25:58.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:58.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:25:58.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430211 2026-02-24T17:25:58.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430211 2026-02-24T17:25:58.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-167503724550 2-206158430211' 2026-02-24T17:25:58.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:25:58.458 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:25:58.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182410 2026-02-24T17:25:58.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182410 2026-02-24T17:25:58.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803788 1-167503724550 2-206158430211 3-107374182410' 2026-02-24T17:25:58.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:25:58.538 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803788 2026-02-24T17:25:58.538 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:25:58.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:25:58.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803788 2026-02-24T17:25:58.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:25:58.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803788 2026-02-24T17:25:58.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-24T17:25:58.541 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803788 2026-02-24T17:25:58.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:58.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-24T17:25:58.769 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:25:59.770 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:25:59.771 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:25:59.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-24T17:25:59.985 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:00.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:26:00.987 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:01.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-24T17:26:01.203 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:01.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-167503724550 2026-02-24T17:26:01.204 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:01.205 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:26:01.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-167503724550 2026-02-24T17:26:01.206 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:01.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724550 2026-02-24T17:26:01.208 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 167503724550' 2026-02-24T17:26:01.208 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 167503724550 2026-02-24T17:26:01.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:01.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-24T17:26:01.430 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:01.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-206158430211 2026-02-24T17:26:01.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:01.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:26:01.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-206158430211 2026-02-24T17:26:01.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:01.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430211 2026-02-24T17:26:01.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 206158430211' 2026-02-24T17:26:01.435 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 206158430211 2026-02-24T17:26:01.435 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:26:01.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430211 -lt 206158430211 2026-02-24T17:26:01.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:01.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182410 2026-02-24T17:26:01.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:01.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:26:01.660 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182410 2026-02-24T17:26:01.660 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:01.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182410 2026-02-24T17:26:01.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182410' 2026-02-24T17:26:01.662 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182410 2026-02-24T17:26:01.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:01.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182410 -lt 107374182410 2026-02-24T17:26:01.884 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:26:01.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:01.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:02.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:26:02.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:26:02.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:26:02.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:26:02.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:26:02.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:26:02.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:26:02.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:26:02.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:26:02.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:26:02.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:02.380 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:02.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:26:02.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:26:02.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:26:02.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:26:02.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:26:02.974 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-1-10 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-1-10 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:26:02.986 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-1-10 td/test-erasure-eio/COPY 2026-02-24T17:26:03.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:26:03.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:26:03.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-73729-1-0 2026-02-24T17:26:03.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-73729-1-0 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-73729-1-0 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:26:03.018 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-73729-1-0 td/test-erasure-eio/ORIGINAL 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-73729-1-0 td/test-erasure-eio 1 0 set 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-1-0 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:26:03.043 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-1-0 2026-02-24T17:26:03.044 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:26:03.044 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-1-0 2026-02-24T17:26:03.044 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-1-0 2026-02-24T17:26:03.044 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:26:03.253 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:26:03.253 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:03.253 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:26:03.253 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:26:03.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:26:03.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:26:03.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-24T17:26:03.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:26:03.544 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:26:03.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:26:03.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-24T17:26:03.555 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:26:03.556 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:26:03.557 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:03.557 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:03.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:03.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:03.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:03.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:03.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:03.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:26:03.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:26:03.867 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:26:03.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:26:03.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:26:03.868 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:04.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:26:04.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:04.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:26:04.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:26:04.183 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:26:04.184 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:04.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:04.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:04.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:04.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:26:04.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:26:04.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:26:04.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:26:04.187 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:26:04.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:04.187 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:26:04.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:26:04.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:26:04.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:26:04.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:26:04.199 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:04.202+0000 7f34528dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:04.201 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:04.206+0000 7f34528dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:04.201 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:04.206+0000 7f34528dda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:04.407 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:26:04.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:04.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:04.589 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:04.594+0000 7f34528dda40 -1 Falling back to public interface 2026-02-24T17:26:04.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:05.181 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:05.186+0000 7f34528dda40 -1 osd.1 52 log_to_monitors true 2026-02-24T17:26:05.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:05.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:05.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:05.624 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:05.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:05.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:05.835 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 58 up_thru 58 down_at 55 last_clean_interval [39,52) [v2:127.0.0.1:6810/4089281705,v1:127.0.0.1:6811/4089281705] [v2:127.0.0.1:6812/4089281705,v1:127.0.0.1:6813/4089281705] exists,up 05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:26:05.835 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:05.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:05.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:05.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:26:05.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:26:05.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:26:05.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:26:05.836 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:26:05.837 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:26:05.837 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:26:05.838 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:26:05.838 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:26:05.838 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:26:05.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:26:05.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:26:05.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:26:05.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:26:05.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:26:05.940 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:26:06.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:26:06.161 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:06.161 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:26:06.161 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:26:06.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:26:06.161 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:06.161 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:26:06.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803792 2026-02-24T17:26:06.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803792 2026-02-24T17:26:06.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792' 2026-02-24T17:26:06.240 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:06.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:26:06.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=249108103171 2026-02-24T17:26:06.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 249108103171 2026-02-24T17:26:06.318 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-249108103171' 2026-02-24T17:26:06.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:06.319 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:26:06.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=206158430214 2026-02-24T17:26:06.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 206158430214 2026-02-24T17:26:06.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-249108103171 2-206158430214' 2026-02-24T17:26:06.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:06.397 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:26:06.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182414 2026-02-24T17:26:06.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182414 2026-02-24T17:26:06.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803792 1-249108103171 2-206158430214 3-107374182414' 2026-02-24T17:26:06.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:06.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803792 2026-02-24T17:26:06.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:06.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:26:06.476 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803792 2026-02-24T17:26:06.476 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:06.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803792 2026-02-24T17:26:06.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-24T17:26:06.478 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803792 2026-02-24T17:26:06.478 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:06.707 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-24T17:26:06.707 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:07.708 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:26:07.708 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:07.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803790 -lt 25769803792 2026-02-24T17:26:07.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:08.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:26:08.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:09.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-24T17:26:09.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:09.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-249108103171 2026-02-24T17:26:09.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:09.144 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:26:09.145 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-249108103171 2026-02-24T17:26:09.146 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:09.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=249108103171 2026-02-24T17:26:09.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 249108103171' 2026-02-24T17:26:09.147 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 249108103171 2026-02-24T17:26:09.147 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:09.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 249108103171 -lt 249108103171 2026-02-24T17:26:09.376 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:09.376 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-206158430214 2026-02-24T17:26:09.377 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:09.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:26:09.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-206158430214 2026-02-24T17:26:09.379 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:09.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=206158430214 2026-02-24T17:26:09.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 206158430214' 2026-02-24T17:26:09.380 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 206158430214 2026-02-24T17:26:09.381 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:26:09.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 206158430215 -lt 206158430214 2026-02-24T17:26:09.589 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:09.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182414 2026-02-24T17:26:09.590 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:09.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:26:09.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182414 2026-02-24T17:26:09.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:09.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182414 2026-02-24T17:26:09.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182414' 2026-02-24T17:26:09.593 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182414 2026-02-24T17:26:09.594 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:09.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182414 -lt 107374182414 2026-02-24T17:26:09.814 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:26:09.815 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:09.815 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:10.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:26:10.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:26:10.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:26:10.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:26:10.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:26:10.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:26:10.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:26:10.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:26:10.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:26:10.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:26:10.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:10.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:10.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:26:10.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:26:10.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:26:10.597 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:26:10.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:26:10.913 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:26:10.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-1-0 2026-02-24T17:26:10.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:26:10.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:26:10.927 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-1-0 2026-02-24T17:26:10.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:26:10.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:26:10.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-1-0 td/test-erasure-eio/COPY 2026-02-24T17:26:10.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:26:10.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:26:10.957 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-24T17:26:10.958 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-73729-1-0 td/test-erasure-eio 2 0 set 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-1-0 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-24T17:26:10.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:26:10.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-1-0 2026-02-24T17:26:10.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:26:10.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-1-0 2026-02-24T17:26:10.960 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-1-0 2026-02-24T17:26:10.960 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:26:11.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:26:11.178 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:11.179 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:26:11.179 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:26:11.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:26:11.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:26:11.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-24T17:26:11.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:26:11.486 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:26:11.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-24T17:26:11.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-24T17:26:11.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-24T17:26:11.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:11.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:26:11.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:26:11.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:11.501 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:11.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:11.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:11.502 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:11.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:11.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:11.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:26:11.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:26:11.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-24T17:26:11.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:26:11.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-24T17:26:11.811 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-73729-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:12.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-24T17:26:12.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:12.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:26:12.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:26:12.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:26:12.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:26:12.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:12.071 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:12.072 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:12.072 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:12.072 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:12.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:12.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:12.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:12.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:12.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:12.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:12.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:12.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:start osd.2 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:26:12.075 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:26:12.093 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:12.098+0000 7f7d9e97ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:12.095 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:12.098+0000 7f7d9e97ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:12.096 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:12.102+0000 7f7d9e97ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:12.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:12.429 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:12.434+0000 7f7d9e97ea40 -1 Falling back to public interface 2026-02-24T17:26:12.499 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:13.074 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:13.078+0000 7f7d9e97ea40 -1 osd.2 63 log_to_monitors true 2026-02-24T17:26:13.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:13.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:13.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:13.500 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:13.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:13.501 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:13.720 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 up in weight 1 up_from 67 up_thru 55 down_at 64 last_clean_interval [48,63) [v2:127.0.0.1:6818/1419379082,v1:127.0.0.1:6819/1419379082] [v2:127.0.0.1:6820/1419379082,v1:127.0.0.1:6821/1419379082] exists,up 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:26:13.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:13.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:13.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:13.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:26:13.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:26:13.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:26:13.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:26:13.721 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:26:13.722 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:26:13.722 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:26:13.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:26:13.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:26:13.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:26:13.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:26:13.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:26:13.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:26:13.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:26:13.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:26:13.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:26:14.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:26:14.046 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:14.046 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:26:14.046 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:26:14.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:26:14.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:14.047 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:26:14.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803795 2026-02-24T17:26:14.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803795 2026-02-24T17:26:14.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795' 2026-02-24T17:26:14.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:14.126 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:26:14.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=249108103174 2026-02-24T17:26:14.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 249108103174 2026-02-24T17:26:14.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-249108103174' 2026-02-24T17:26:14.204 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:14.205 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:26:14.282 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=287762808835 2026-02-24T17:26:14.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 287762808835 2026-02-24T17:26:14.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-249108103174 2-287762808835' 2026-02-24T17:26:14.283 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:14.283 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:26:14.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182417 2026-02-24T17:26:14.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182417 2026-02-24T17:26:14.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803795 1-249108103174 2-287762808835 3-107374182417' 2026-02-24T17:26:14.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:14.363 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803795 2026-02-24T17:26:14.363 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:14.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:26:14.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803795 2026-02-24T17:26:14.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:14.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803795 2026-02-24T17:26:14.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803795' 2026-02-24T17:26:14.366 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803795 2026-02-24T17:26:14.366 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:14.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803795 -lt 25769803795 2026-02-24T17:26:14.591 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:14.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-249108103174 2026-02-24T17:26:14.592 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:14.593 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:26:14.594 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-249108103174 2026-02-24T17:26:14.594 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:14.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=249108103174 2026-02-24T17:26:14.596 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 249108103174' 2026-02-24T17:26:14.596 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 249108103174 2026-02-24T17:26:14.596 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:14.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 249108103172 -lt 249108103174 2026-02-24T17:26:14.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:15.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:26:15.815 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:16.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 249108103172 -lt 249108103174 2026-02-24T17:26:16.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:17.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:26:17.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:17.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 249108103175 -lt 249108103174 2026-02-24T17:26:17.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:17.243 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-287762808835 2026-02-24T17:26:17.243 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:17.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:26:17.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-287762808835 2026-02-24T17:26:17.245 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:17.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=287762808835 2026-02-24T17:26:17.246 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 287762808835' 2026-02-24T17:26:17.246 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 287762808835 2026-02-24T17:26:17.246 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:26:17.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 287762808835 -lt 287762808835 2026-02-24T17:26:17.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:17.476 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182417 2026-02-24T17:26:17.476 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:17.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:26:17.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182417 2026-02-24T17:26:17.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:17.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182417 2026-02-24T17:26:17.479 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182417' 2026-02-24T17:26:17.479 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182417 2026-02-24T17:26:17.479 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:17.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182417 -lt 107374182417 2026-02-24T17:26:17.707 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:26:17.707 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:17.707 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:18.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:26:18.003 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:26:18.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:26:18.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:26:18.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:26:18.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:26:18.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:26:18.003 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:26:18.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:26:18.214 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:26:18.214 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:18.214 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:18.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:26:18.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:26:18.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:26:18.492 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:26:18.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:26:18.815 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:26:18.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-24T17:26:18.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-1-0 2026-02-24T17:26:18.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:26:18.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:26:18.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-1-0 2026-02-24T17:26:18.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:26:18.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:26:18.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-1-0 td/test-erasure-eio/COPY 2026-02-24T17:26:18.852 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:26:18.853 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:26:18.854 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:26:18.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-24T17:26:18.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-24T17:26:18.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-24T17:26:18.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-24T17:26:18.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-24T17:26:18.857 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-73729-1-256 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-73729-1-256 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-73729-1-256 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:26:18.858 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:18.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:26:18.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:18.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:26:18.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:26:18.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:26:18.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-73729-1-256 td/test-erasure-eio/ORIGINAL 2026-02-24T17:26:18.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-73729-1-256 td/test-erasure-eio 1 256 add 2026-02-24T17:26:18.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-1-256 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-1-256 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:26:18.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-1-256 2026-02-24T17:26:18.882 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-1-256 2026-02-24T17:26:18.882 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-24T17:26:19.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:26:19.408 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:19.424 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:19.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:19.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:19.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:19.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:26:19.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:26:19.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:26:19.733 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:26:19.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:26:19.734 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:19.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:19.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:19.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:26:19.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:26:19.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:26:19.911 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:26:19.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:19.911 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:26:19.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:26:19.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:26:19.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:26:19.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:26:19.926 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:19.930+0000 7f6c79638a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:19.927 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:19.930+0000 7f6c79638a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:19.928 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:19.934+0000 7f6c79638a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:20.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:26:20.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:20.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:20.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:20.389 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:20.393+0000 7f6c79638a40 -1 Falling back to public interface 2026-02-24T17:26:20.983 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:20.985+0000 7f6c79638a40 -1 osd.1 71 log_to_monitors true 2026-02-24T17:26:21.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:21.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:21.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:21.342 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:21.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:21.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 76 up_thru 76 down_at 73 last_clean_interval [58,71) [v2:127.0.0.1:6810/2713706967,v1:127.0.0.1:6811/2713706967] [v2:127.0.0.1:6812/2713706967,v1:127.0.0.1:6813/2713706967] exists,up 05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:26:21.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:26:21.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:26:21.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:26:21.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:26:21.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:26:21.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:26:21.658 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:26:21.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:26:21.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:26:21.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:26:21.659 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:26:21.659 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:26:21.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:26:21.869 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:21.869 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:26:21.869 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:26:21.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:26:21.870 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:21.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:26:21.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803799 2026-02-24T17:26:21.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803799 2026-02-24T17:26:21.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799' 2026-02-24T17:26:21.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:21.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:26:22.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=326417514499 2026-02-24T17:26:22.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 326417514499 2026-02-24T17:26:22.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-326417514499' 2026-02-24T17:26:22.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:22.027 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:26:22.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=287762808838 2026-02-24T17:26:22.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 287762808838 2026-02-24T17:26:22.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-326417514499 2-287762808838' 2026-02-24T17:26:22.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:22.105 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:26:22.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182421 2026-02-24T17:26:22.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182421 2026-02-24T17:26:22.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803799 1-326417514499 2-287762808838 3-107374182421' 2026-02-24T17:26:22.187 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:22.187 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803799 2026-02-24T17:26:22.187 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:22.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:26:22.189 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803799 2026-02-24T17:26:22.189 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:22.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803799 2026-02-24T17:26:22.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803799' 2026-02-24T17:26:22.190 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803799 2026-02-24T17:26:22.191 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:22.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803799 -lt 25769803799 2026-02-24T17:26:22.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:22.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-326417514499 2026-02-24T17:26:22.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:22.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:26:22.417 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-326417514499 2026-02-24T17:26:22.418 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:22.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=326417514499 2026-02-24T17:26:22.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 326417514499' 2026-02-24T17:26:22.419 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 326417514499 2026-02-24T17:26:22.420 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:22.650 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 326417514499 -lt 326417514499 2026-02-24T17:26:22.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:22.651 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-287762808838 2026-02-24T17:26:22.652 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:22.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:26:22.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-287762808838 2026-02-24T17:26:22.653 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:22.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=287762808838 2026-02-24T17:26:22.655 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 287762808838' 2026-02-24T17:26:22.655 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 287762808838 2026-02-24T17:26:22.655 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:26:22.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 287762808838 -lt 287762808838 2026-02-24T17:26:22.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:22.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182421 2026-02-24T17:26:22.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:22.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:26:22.883 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182421 2026-02-24T17:26:22.883 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:22.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182421 2026-02-24T17:26:22.885 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182421' 2026-02-24T17:26:22.885 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182421 2026-02-24T17:26:22.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:23.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182419 -lt 107374182421 2026-02-24T17:26:23.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:24.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:26:24.096 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:24.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182421 -lt 107374182421 2026-02-24T17:26:24.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:26:24.310 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:24.310 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:24.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:26:24.598 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:26:24.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:26:24.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:26:24.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:26:24.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:26:24.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:26:24.599 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:26:24.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:26:24.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:26:24.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:24.816 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:25.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:26:25.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:26:25.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:26:25.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-24T17:26:25.101 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:26:25.101 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:256 bytes copied, 3.9323e-05 s, 6.5 MB/s 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:26:25.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:26:25.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:26:25.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:25.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:25.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:25.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:25.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:25.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:25.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:25.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:26:25.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:26:25.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:26:25.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:26:25.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:26:25.410 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:25.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:26:25.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:25.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:26:25.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:26:25.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:26:25.726 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:26:25.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:25.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:26:25.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:26:25.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:26:25.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:26:25.727 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:26:25.741 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:25.745+0000 7faf571baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:25.743 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:25.745+0000 7faf571baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:25.744 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:25.749+0000 7faf571baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:25.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:25.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:26.153 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:26.214 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:26.217+0000 7faf571baa40 -1 Falling back to public interface 2026-02-24T17:26:26.850 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:26.853+0000 7faf571baa40 -1 osd.1 77 log_to_monitors true 2026-02-24T17:26:27.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:27.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:27.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:27.155 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:27.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:27.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:27.370 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 81 up_thru 81 down_at 78 last_clean_interval [76,77) [v2:127.0.0.1:6810/1089745420,v1:127.0.0.1:6811/1089745420] [v2:127.0.0.1:6812/1089745420,v1:127.0.0.1:6813/1089745420] exists,up 05cc947f-6450-4d33-abd9-18307b51f708 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:26:27.371 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:26:27.372 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:26:27.372 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:26:27.372 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:26:27.372 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:26:27.372 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:26:27.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:26:27.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:26:27.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:26:27.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:26:27.483 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:26:27.483 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:26:27.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:26:27.696 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:27.696 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:26:27.696 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:26:27.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:26:27.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:27.696 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:26:27.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803802 2026-02-24T17:26:27.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803802 2026-02-24T17:26:27.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802' 2026-02-24T17:26:27.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:27.779 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:26:27.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=347892350979 2026-02-24T17:26:27.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 347892350979 2026-02-24T17:26:27.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-347892350979' 2026-02-24T17:26:27.856 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:27.856 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:26:27.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=287762808841 2026-02-24T17:26:27.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 287762808841 2026-02-24T17:26:27.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-347892350979 2-287762808841' 2026-02-24T17:26:27.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:27.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:26:28.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182424 2026-02-24T17:26:28.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182424 2026-02-24T17:26:28.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803802 1-347892350979 2-287762808841 3-107374182424' 2026-02-24T17:26:28.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:28.014 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803802 2026-02-24T17:26:28.014 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:28.015 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:26:28.015 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803802 2026-02-24T17:26:28.016 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:28.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803802 2026-02-24T17:26:28.017 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803802' 2026-02-24T17:26:28.017 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803802 2026-02-24T17:26:28.018 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:28.237 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803802 -lt 25769803802 2026-02-24T17:26:28.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:28.238 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-347892350979 2026-02-24T17:26:28.239 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:28.239 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:26:28.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-347892350979 2026-02-24T17:26:28.240 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:28.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=347892350979 2026-02-24T17:26:28.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 347892350979' 2026-02-24T17:26:28.242 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 347892350979 2026-02-24T17:26:28.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:28.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 347892350979 -lt 347892350979 2026-02-24T17:26:28.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:28.471 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-287762808841 2026-02-24T17:26:28.471 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:28.472 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:26:28.473 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-287762808841 2026-02-24T17:26:28.473 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:28.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=287762808841 2026-02-24T17:26:28.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 287762808841' 2026-02-24T17:26:28.474 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 287762808841 2026-02-24T17:26:28.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:26:28.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 287762808842 -lt 287762808841 2026-02-24T17:26:28.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:28.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182424 2026-02-24T17:26:28.697 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:28.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:26:28.699 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182424 2026-02-24T17:26:28.699 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:28.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182424 2026-02-24T17:26:28.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182424' 2026-02-24T17:26:28.701 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182424 2026-02-24T17:26:28.701 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:28.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182424 -lt 107374182424 2026-02-24T17:26:28.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:26:28.924 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:28.925 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:29.215 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:26:29.216 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:26:29.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:26:29.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:26:29.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:26:29.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:26:29.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:26:29.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:26:29.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:26:29.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:26:29.431 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:29.432 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:29.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:26:29.721 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:26:29.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:26:29.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:26:29.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:26:30.037 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:26:30.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-1-256 2026-02-24T17:26:30.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:26:30.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:26:30.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-1-256 2026-02-24T17:26:30.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:26:30.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:26:30.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-1-256 td/test-erasure-eio/COPY 2026-02-24T17:26:30.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:26:30.074 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:26:30.076 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-24T17:26:30.078 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-73729-1-256 td/test-erasure-eio 2 256 add 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-73729-1-256 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-73729-1-256 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:26:30.079 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-73729-1-256 2026-02-24T17:26:30.080 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-73729-1-256 2026-02-24T17:26:30.080 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:26:30.297 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:26:30.297 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:30.297 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:26:30.297 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:26:30.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-24T17:26:30.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-24T17:26:30.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-24T17:26:30.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-24T17:26:30.607 INFO:tasks.workunit.client.0.trial066.stderr:noout is set 2026-02-24T17:26:30.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-24T17:26:30.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:30.621 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:30.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:30.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:30.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:30.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:30.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:30.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:26:30.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:26:30.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-24T17:26:30.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:26:30.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-24T17:26:30.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-73729-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:31.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-24T17:26:31.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:31.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:26:31.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:26:31.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:31.094 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:31.095 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:31.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:26:31.096 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:26:31.097 INFO:tasks.workunit.client.0.trial066.stderr:start osd.2 2026-02-24T17:26:31.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:31.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:26:31.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:26:31.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:26:31.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:26:31.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:26:31.117 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:31.121+0000 7f9d56fb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:31.118 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:31.121+0000 7f9d56fb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:31.119 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:31.121+0000 7f9d56fb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:31.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:26:31.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:31.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:26:31.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:31.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:31.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:31.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:31.308 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:26:31.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:31.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:31.526 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:31.574 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:31.577+0000 7f9d56fb7a40 -1 Falling back to public interface 2026-02-24T17:26:32.208 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:32.209+0000 7f9d56fb7a40 -1 osd.2 85 log_to_monitors true 2026-02-24T17:26:32.527 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:32.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:32.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:32.528 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:32.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:32.528 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 up in weight 1 up_from 90 up_thru 78 down_at 87 last_clean_interval [67,85) [v2:127.0.0.1:6818/2244111531,v1:127.0.0.1:6819/2244111531] [v2:127.0.0.1:6820/2244111531,v1:127.0.0.1:6821/2244111531] exists,up 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:26:32.737 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:26:32.738 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:26:32.738 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:26:32.738 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:26:32.739 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:26:32.739 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:26:32.739 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:26:32.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:26:32.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:26:32.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:26:32.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:26:32.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:26:32.844 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:26:33.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:26:33.061 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:33.061 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:26:33.061 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:26:33.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:26:33.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:33.062 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:26:33.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803805 2026-02-24T17:26:33.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803805 2026-02-24T17:26:33.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805' 2026-02-24T17:26:33.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:33.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:26:33.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=347892350982 2026-02-24T17:26:33.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 347892350982 2026-02-24T17:26:33.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-347892350982' 2026-02-24T17:26:33.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:33.223 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:26:33.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=386547056643 2026-02-24T17:26:33.303 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 386547056643 2026-02-24T17:26:33.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-347892350982 2-386547056643' 2026-02-24T17:26:33.304 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:33.304 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:26:33.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182427 2026-02-24T17:26:33.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182427 2026-02-24T17:26:33.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803805 1-347892350982 2-386547056643 3-107374182427' 2026-02-24T17:26:33.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:33.384 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803805 2026-02-24T17:26:33.384 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:33.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:26:33.386 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803805 2026-02-24T17:26:33.386 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:33.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803805 2026-02-24T17:26:33.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803805' 2026-02-24T17:26:33.387 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803805 2026-02-24T17:26:33.387 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:33.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803803 -lt 25769803805 2026-02-24T17:26:33.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:34.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:26:34.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:34.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803805 -lt 25769803805 2026-02-24T17:26:34.823 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:34.824 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-347892350982 2026-02-24T17:26:34.824 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:34.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:26:34.825 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-347892350982 2026-02-24T17:26:34.825 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:34.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=347892350982 2026-02-24T17:26:34.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 347892350982' 2026-02-24T17:26:34.827 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 347892350982 2026-02-24T17:26:34.828 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:35.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 347892350982 -lt 347892350982 2026-02-24T17:26:35.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:35.055 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-386547056643 2026-02-24T17:26:35.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:35.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:26:35.057 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-386547056643 2026-02-24T17:26:35.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:35.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=386547056643 2026-02-24T17:26:35.059 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 386547056643' 2026-02-24T17:26:35.059 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 386547056643 2026-02-24T17:26:35.060 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:26:35.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 386547056643 -lt 386547056643 2026-02-24T17:26:35.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:35.272 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182427 2026-02-24T17:26:35.272 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:35.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:26:35.274 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182427 2026-02-24T17:26:35.274 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:35.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182427 2026-02-24T17:26:35.276 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182427' 2026-02-24T17:26:35.276 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182427 2026-02-24T17:26:35.276 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:35.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182427 -lt 107374182427 2026-02-24T17:26:35.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:26:35.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:35.495 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:35.788 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:26:35.789 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:26:35.789 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:26:35.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:26:35.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:26:35.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:26:35.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:26:35.790 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:26:36.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:26:36.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:26:36.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:36.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:26:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:26:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:26:36.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-24T17:26:36.289 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records in 2026-02-24T17:26:36.289 INFO:tasks.workunit.client.0.trial066.stderr:1+0 records out 2026-02-24T17:26:36.289 INFO:tasks.workunit.client.0.trial066.stderr:256 bytes copied, 4.337e-05 s, 5.9 MB/s 2026-02-24T17:26:36.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:36.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:36.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-24T17:26:36.599 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-73729-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-24T17:26:36.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-24T17:26:36.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:36.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:26:36.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:26:36.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:26:36.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:26:36.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:36.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:26:36.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:26:36.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:26:36.911 INFO:tasks.workunit.client.0.trial066.stderr:start osd.2 2026-02-24T17:26:36.912 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:36.912 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:26:36.913 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:26:36.913 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:26:36.913 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:26:36.913 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:26:36.926 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:36.929+0000 7f86859b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:36.928 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:36.929+0000 7f86859b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:36.929 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:36.933+0000 7f86859b4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:37.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:37.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:37.409 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:37.413+0000 7f86859b4a40 -1 Falling back to public interface 2026-02-24T17:26:38.082 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:38.085+0000 7f86859b4a40 -1 osd.2 91 log_to_monitors true 2026-02-24T17:26:38.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:38.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:38.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:38.341 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:38.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:38.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:38.557 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 up in weight 1 up_from 95 up_thru 78 down_at 92 last_clean_interval [90,91) [v2:127.0.0.1:6818/2870353034,v1:127.0.0.1:6819/2870353034] [v2:127.0.0.1:6820/2870353034,v1:127.0.0.1:6821/2870353034] exists,up 8de4b420-d4d1-4f76-bfd3-ca40b5afdb8a 2026-02-24T17:26:38.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:38.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:38.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:38.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:26:38.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:26:38.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:26:38.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:26:38.558 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:26:38.559 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:26:38.559 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:26:38.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:26:38.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:26:38.559 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:26:38.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:26:38.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:26:38.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:26:38.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:26:38.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:26:38.666 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:26:38.886 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:26:38.887 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:26:38.887 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:26:38.887 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:26:38.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:26:38.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:38.887 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:26:38.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803808 2026-02-24T17:26:38.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803808 2026-02-24T17:26:38.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803808' 2026-02-24T17:26:38.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:38.970 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:26:39.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=347892350985 2026-02-24T17:26:39.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 347892350985 2026-02-24T17:26:39.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803808 1-347892350985' 2026-02-24T17:26:39.052 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:39.053 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:26:39.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=408021893123 2026-02-24T17:26:39.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 408021893123 2026-02-24T17:26:39.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803808 1-347892350985 2-408021893123' 2026-02-24T17:26:39.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:26:39.131 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:26:39.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182430 2026-02-24T17:26:39.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182430 2026-02-24T17:26:39.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803808 1-347892350985 2-408021893123 3-107374182430' 2026-02-24T17:26:39.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:39.214 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803808 2026-02-24T17:26:39.214 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:39.215 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:26:39.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803808 2026-02-24T17:26:39.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:39.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803808 2026-02-24T17:26:39.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803808' 2026-02-24T17:26:39.217 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803808 2026-02-24T17:26:39.218 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:39.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803806 -lt 25769803808 2026-02-24T17:26:39.439 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:26:40.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:26:40.441 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:26:40.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803808 -lt 25769803808 2026-02-24T17:26:40.660 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:40.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-347892350985 2026-02-24T17:26:40.661 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:40.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:26:40.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-347892350985 2026-02-24T17:26:40.663 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:40.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=347892350985 2026-02-24T17:26:40.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 347892350985' 2026-02-24T17:26:40.664 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 347892350985 2026-02-24T17:26:40.665 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:26:40.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 347892350985 -lt 347892350985 2026-02-24T17:26:40.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:40.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-408021893123 2026-02-24T17:26:40.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:40.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:26:40.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-408021893123 2026-02-24T17:26:40.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:40.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=408021893123 2026-02-24T17:26:40.882 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 408021893123' 2026-02-24T17:26:40.882 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 408021893123 2026-02-24T17:26:40.883 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:26:41.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 408021893123 -lt 408021893123 2026-02-24T17:26:41.106 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:26:41.106 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182430 2026-02-24T17:26:41.107 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:26:41.108 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:26:41.109 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182430 2026-02-24T17:26:41.109 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:26:41.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182430 2026-02-24T17:26:41.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182430' 2026-02-24T17:26:41.110 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 107374182430 2026-02-24T17:26:41.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:26:41.331 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182430 -lt 107374182430 2026-02-24T17:26:41.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:26:41.332 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:41.333 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:41.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:26:41.606 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:26:41.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:26:41.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:26:41.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:26:41.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:26:41.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:26:41.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:26:41.821 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:26:41.821 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:26:41.822 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:26:41.822 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:26:42.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:26:42.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:26:42.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:26:42.120 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-24T17:26:42.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-24T17:26:42.439 INFO:tasks.workunit.client.0.trial066.stderr:noout is unset 2026-02-24T17:26:42.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-24T17:26:42.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-73729-1-256 2026-02-24T17:26:42.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:26:42.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:26:42.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-73729-1-256 2026-02-24T17:26:42.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:26:42.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:26:42.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-73729-1-256 td/test-erasure-eio/COPY 2026-02-24T17:26:42.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:26:42.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:26:42.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:26:42.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:26:42.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:26:42.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:26:42.693 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:26:42.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:26:42.972 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:26:42.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:26:42.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:26:42.981 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:26:42.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:26:42.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:42.982 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:42.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:42.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:42.982 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:43.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:43.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:26:43.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:26:43.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:26:43.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:26:43.097 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:26:43.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:26:43.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:26:43.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:26:43.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:26:43.099 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:26:43.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:26:43.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:26:43.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:26:43.126 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:26:43.126 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:43.126 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:43.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:26:43.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:26:43.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:26:43.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:26:43.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:26:43.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:26:43.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:26:43.131 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:26:43.132 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:26:43.133 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:26:43.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:26:43.134 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:26:43.135 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:26:43.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:26:43.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:26:43.136 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:26:43.137 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:26:43.138 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:26:43.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:26:43.139 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:26:43.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:26:43.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:43.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:43.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:26:43.141 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:26:43.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:26:43.142 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:26:43.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:26:43.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:43.143 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:43.143 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:26:43.145 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:26:43.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:26:43.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:26:43.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:26:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:26:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:26:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:26:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:26:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:26:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:26:43.146 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:26:43.167 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:26:43.167 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:43.167 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:43.167 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:43.168 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:43.168 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:43.168 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:43.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:26:43.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:26:43.197 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:26:43.197 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:26:43.197 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:26:43.197 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:26:43.197 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:26:43.198 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:26:43.198 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:26:43.198 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:26:43.198 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:26:43.198 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:43.199 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:43.199 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:26:43.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:26:43.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:26:43.254 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:26:43.254 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:26:43.255 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:26:43.255 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:26:43.255 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:26:43.255 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:26:43.255 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:26:43.255 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:26:43.255 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:26:43.256 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:43.256 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:43.256 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:26:43.256 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:26:43.256 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:26:43.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:26:43.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:26:43.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:26:43.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:26:43.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:26:43.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:26:43.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:26:43.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:26:43.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:43.421 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:43.422 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:43.422 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:43.422 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:43.422 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:43.423 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:26:43.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:26:43.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:26:43.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:26:43.551 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:26:43.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:26:44.561 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:26:44.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:26:44.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:26:44.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:26:44.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:44.562 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:44.562 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:26:44.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:26:44.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:26:44.610 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:26:44.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:26:44.618 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:26:43.189+0000 7faf4707bec0 0 load: jerasure load: lrc 2026-02-24T17:26:44.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-24T17:26:44.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-24T17:26:44.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-24T17:26:44.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-24T17:26:44.619 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:26:44.620 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-24T17:26:44.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-24T17:26:44.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:26:44.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:26:44.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:44.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:26:44.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:26:44.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:26:44.625 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:26:44.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:26:44.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:26:44.628 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:26:44.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c92a71a8-9bab-454e-b1a5-e74950ec2c7a 2026-02-24T17:26:44.628 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 c92a71a8-9bab-454e-b1a5-e74950ec2c7a 2026-02-24T17:26:44.629 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 c92a71a8-9bab-454e-b1a5-e74950ec2c7a' 2026-02-24T17:26:44.629 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:26:44.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBU351pRMR3JhAAttIgsPHlnMZU57MhSD4j7w== 2026-02-24T17:26:44.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBU351pRMR3JhAAttIgsPHlnMZU57MhSD4j7w=="}' 2026-02-24T17:26:44.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c92a71a8-9bab-454e-b1a5-e74950ec2c7a -i td/test-erasure-eio/0/new.json 2026-02-24T17:26:44.748 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:26:44.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:26:44.757 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBU351pRMR3JhAAttIgsPHlnMZU57MhSD4j7w== --osd-uuid c92a71a8-9bab-454e-b1a5-e74950ec2c7a 2026-02-24T17:26:44.774 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:44.777+0000 7f1d38d79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:44.775 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:44.777+0000 7f1d38d79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:44.776 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:44.781+0000 7f1d38d79a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:44.777 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:44.781+0000 7f1d38d79a40 -1 bdev(0x55b1c5516400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:26:44.777 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:44.781+0000 7f1d38d79a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:26:45.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:26:45.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:26:45.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:26:45.489 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:26:45.489 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:26:45.644 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:26:45.645 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:26:45.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:26:45.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:26:45.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:26:45.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:45.663 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:45.665+0000 7f88e850ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:45.665 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:45.669+0000 7f88e850ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:45.666 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:45.669+0000 7f88e850ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:45.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:26:45.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:45.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:26:45.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:45.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:45.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:45.756 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:26:45.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:45.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:45.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:26:45.971 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:46.082 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:46.085+0000 7f88e850ea40 -1 Falling back to public interface 2026-02-24T17:26:46.629 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:46.633+0000 7f88e850ea40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:26:46.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:46.973 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:46.973 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:26:46.974 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:46.974 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:46.974 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:26:47.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:48.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:48.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:48.189 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:26:48.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:26:48.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:48.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:26:48.411 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/263513047,v1:127.0.0.1:6803/263513047] [v2:127.0.0.1:6804/263513047,v1:127.0.0.1:6805/263513047] exists,up c92a71a8-9bab-454e-b1a5-e74950ec2c7a 2026-02-24T17:26:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:26:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:26:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:48.415 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:26:48.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:48.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:26:48.418 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:26:48.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:26:48.419 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:26:48.419 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=372d23cd-edde-43a5-8bcf-1a734e531b20 2026-02-24T17:26:48.420 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 372d23cd-edde-43a5-8bcf-1a734e531b20 2026-02-24T17:26:48.420 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 372d23cd-edde-43a5-8bcf-1a734e531b20' 2026-02-24T17:26:48.420 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:26:48.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBY351pvyn3GRAAiol8sXc0tR5LxJrpGmXiNg== 2026-02-24T17:26:48.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBY351pvyn3GRAAiol8sXc0tR5LxJrpGmXiNg=="}' 2026-02-24T17:26:48.428 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 372d23cd-edde-43a5-8bcf-1a734e531b20 -i td/test-erasure-eio/1/new.json 2026-02-24T17:26:48.642 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:26:48.652 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:26:48.653 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBY351pvyn3GRAAiol8sXc0tR5LxJrpGmXiNg== --osd-uuid 372d23cd-edde-43a5-8bcf-1a734e531b20 2026-02-24T17:26:48.670 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:48.673+0000 7fa6d0466a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:48.671 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:48.673+0000 7fa6d0466a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:48.672 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:48.677+0000 7fa6d0466a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:48.673 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:48.677+0000 7fa6d0466a40 -1 bdev(0x55c1b90f8400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:26:48.673 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:48.677+0000 7fa6d0466a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:26:49.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:26:49.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:26:49.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:26:49.311 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:26:49.311 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:26:49.615 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:26:49.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:26:49.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:49.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:26:49.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:26:49.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:26:49.640 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:49.645+0000 7fa9b00fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:49.642 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:49.645+0000 7fa9b00fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:49.643 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:49.645+0000 7fa9b00fea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:49.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:26:49.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:49.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:26:49.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:49.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:49.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:49.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:49.829 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:26:49.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:49.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:50.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:50.094 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:50.097+0000 7fa9b00fea40 -1 Falling back to public interface 2026-02-24T17:26:50.681 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:50.685+0000 7fa9b00fea40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:26:51.042 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:51.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:51.043 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:26:51.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:51.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:51.044 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:51.256 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:52.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:52.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:52.257 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:26:52.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:26:52.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:52.257 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:26:52.470 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/410437957,v1:127.0.0.1:6811/410437957] [v2:127.0.0.1:6812/410437957,v1:127.0.0.1:6813/410437957] exists,up 372d23cd-edde-43a5-8bcf-1a734e531b20 2026-02-24T17:26:52.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:52.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:52.473 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:52.474 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:52.475 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:26:52.476 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:26:52.477 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:26:52.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d78f2ce0-d6b0-4a9f-9244-2bf4f0818e23 2026-02-24T17:26:52.477 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d78f2ce0-d6b0-4a9f-9244-2bf4f0818e23' 2026-02-24T17:26:52.478 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 d78f2ce0-d6b0-4a9f-9244-2bf4f0818e23 2026-02-24T17:26:52.478 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:26:52.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBc351p/iB/HRAAnV1zNHu+BWc0v6E71YS8vg== 2026-02-24T17:26:52.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBc351p/iB/HRAAnV1zNHu+BWc0v6E71YS8vg=="}' 2026-02-24T17:26:52.487 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d78f2ce0-d6b0-4a9f-9244-2bf4f0818e23 -i td/test-erasure-eio/2/new.json 2026-02-24T17:26:52.708 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:26:52.719 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:26:52.720 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBc351p/iB/HRAAnV1zNHu+BWc0v6E71YS8vg== --osd-uuid d78f2ce0-d6b0-4a9f-9244-2bf4f0818e23 2026-02-24T17:26:52.737 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:52.741+0000 7f3fe67c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:52.738 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:52.741+0000 7f3fe67c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:52.739 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:52.741+0000 7f3fe67c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:52.740 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:52.745+0000 7f3fe67c2a40 -1 bdev(0x5622b7bd4400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:26:52.740 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:52.745+0000 7f3fe67c2a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:26:53.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:26:53.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:26:53.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:26:53.392 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:26:53.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:26:53.694 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:26:53.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:26:53.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:53.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:26:53.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:26:53.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:26:53.717 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:53.721+0000 7fda88c1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:53.719 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:53.721+0000 7fda88c1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:53.720 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:53.725+0000 7fda88c1ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:53.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:54.090 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:54.093+0000 7fda88c1ca40 -1 Falling back to public interface 2026-02-24T17:26:54.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:54.690 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:54.693+0000 7fda88c1ca40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:26:55.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:55.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:55.127 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:26:55.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:55.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:55.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:26:55.339 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1138186064,v1:127.0.0.1:6819/1138186064] [v2:127.0.0.1:6820/1138186064,v1:127.0.0.1:6821/1138186064] exists,up d78f2ce0-d6b0-4a9f-9244-2bf4f0818e23 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:26:55.343 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:26:55.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:26:55.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:26:55.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:26:55.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:26:55.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:26:55.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:26:55.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:26:55.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:26:55.346 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:26:55.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2786afe1-3e9d-4881-a193-476c8ec963f6 2026-02-24T17:26:55.347 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 2786afe1-3e9d-4881-a193-476c8ec963f6 2026-02-24T17:26:55.348 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2786afe1-3e9d-4881-a193-476c8ec963f6' 2026-02-24T17:26:55.348 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:26:55.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBf351psh2kFRAAj3OfFAYncY4Vnxwcakm92w== 2026-02-24T17:26:55.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBf351psh2kFRAAj3OfFAYncY4Vnxwcakm92w=="}' 2026-02-24T17:26:55.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2786afe1-3e9d-4881-a193-476c8ec963f6 -i td/test-erasure-eio/3/new.json 2026-02-24T17:26:55.575 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:26:55.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:26:55.585 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBf351psh2kFRAAj3OfFAYncY4Vnxwcakm92w== --osd-uuid 2786afe1-3e9d-4881-a193-476c8ec963f6 2026-02-24T17:26:55.602 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:55.605+0000 7fe1d9294a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:55.604 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:55.605+0000 7fe1d9294a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:55.605 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:55.609+0000 7fe1d9294a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:55.606 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:55.609+0000 7fe1d9294a40 -1 bdev(0x55946848e400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:26:55.606 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:55.609+0000 7fe1d9294a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:26:56.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:26:56.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:26:56.270 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:26:56.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:26:56.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:26:56.570 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:26:56.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:26:56.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:26:56.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:26:56.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:26:56.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:26:56.594 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:56.597+0000 7f5c3b234a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:56.598 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:56.601+0000 7f5c3b234a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:56.598 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:56.601+0000 7f5c3b234a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:26:56.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:56.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:26:56.966 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:56.969+0000 7f5c3b234a40 -1 Falling back to public interface 2026-02-24T17:26:56.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:57.595 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:26:57.597+0000 7f5c3b234a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:26:57.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:57.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:57.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:26:57.993 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:26:57.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:57.994 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:26:58.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:26:59.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:26:59.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:26:59.223 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:26:59.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:26:59.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:26:59.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:26:59.443 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1178923223,v1:127.0.0.1:6827/1178923223] [v2:127.0.0.1:6828/1178923223,v1:127.0.0.1:6829/1178923223] exists,up 2786afe1-3e9d-4881-a193-476c8ec963f6 2026-02-24T17:26:59.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:26:59.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:26:59.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:26:59.444 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:26:59.444 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:26:59.444 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:26:59.444 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:26:59.444 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:26:59.445 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:26:59.445 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:26:59.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:26:59.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:26:59.502 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:26:59.507 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:26:59.510 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:26:46.089+0000 7f88e850ea40 0 load: jerasure load: lrc 2026-02-24T17:26:59.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-24T17:26:59.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-24T17:26:59.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:26:59.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:26:59.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-24T17:26:59.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:26:59.511 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-24T17:26:59.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:26:59.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-24T17:26:59.790 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:26:59.790 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:27:00.164 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:27:00.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:27:01.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:27:01.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:27:01.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:27:01.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:27:01.177 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:27:01.178 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:27:01.178 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:27:01.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:27:01.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:27:01.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:27:01.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:27:01.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:27:01.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:27:01.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:27:01.286 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:27:01.286 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:27:01.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:27:01.496 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:01.497 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:27:01.497 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:27:01.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:27:01.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:01.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:27:01.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-24T17:27:01.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-24T17:27:01.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-24T17:27:01.576 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:01.577 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:27:01.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-24T17:27:01.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-24T17:27:01.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-24T17:27:01.656 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:01.656 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:27:01.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378628 2026-02-24T17:27:01.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378628 2026-02-24T17:27:01.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378628' 2026-02-24T17:27:01.735 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:01.735 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:27:01.812 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=115964116995 2026-02-24T17:27:01.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 115964116995 2026-02-24T17:27:01.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378628 3-115964116995' 2026-02-24T17:27:01.813 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:01.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-24T17:27:01.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:01.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:27:01.815 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-24T17:27:01.815 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:01.817 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-24T17:27:01.817 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-24T17:27:01.817 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803781 2026-02-24T17:27:01.817 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:02.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-24T17:27:02.034 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:02.034 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574852 2026-02-24T17:27:02.034 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:02.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:27:02.036 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574852 2026-02-24T17:27:02.036 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:02.037 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-24T17:27:02.038 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 55834574852 2026-02-24T17:27:02.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-24T17:27:02.038 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:27:02.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-24T17:27:02.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:02.262 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378628 2026-02-24T17:27:02.263 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:02.263 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:27:02.264 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378628 2026-02-24T17:27:02.264 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:02.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378628 2026-02-24T17:27:02.266 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-24T17:27:02.266 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 81604378628 2026-02-24T17:27:02.267 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:27:02.480 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378626 -lt 81604378628 2026-02-24T17:27:02.480 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:27:03.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:27:03.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:27:03.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378626 -lt 81604378628 2026-02-24T17:27:03.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:27:04.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:27:04.700 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:27:04.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-24T17:27:04.917 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:04.918 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-115964116995 2026-02-24T17:27:04.918 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:04.919 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:27:04.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-115964116995 2026-02-24T17:27:04.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:04.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=115964116995 2026-02-24T17:27:04.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-24T17:27:04.922 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 115964116995 2026-02-24T17:27:04.922 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:27:05.140 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-24T17:27:05.140 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:27:05.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:05.141 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:05.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:27:05.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:27:05.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:27:05.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:27:05.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:27:05.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:27:05.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:27:05.437 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:27:05.646 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:27:05.647 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:27:05.647 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:05.647 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-24T17:27:05.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-73729-0 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-73729-0 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-73729-0 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:27:05.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-73729-0 td/test-erasure-eio/ORIGINAL 2026-02-24T17:27:05.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-73729-0 td/test-erasure-eio 0 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-73729-0 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-73729-0 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-0 2026-02-24T17:27:05.954 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-0 2026-02-24T17:27:05.955 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:27:06.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:27:06.174 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:06.174 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:27:06.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:27:06.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-24T17:27:06.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:27:06.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-24T17:27:06.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:27:06.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:27:06.177 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-24T17:27:06.177 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:27:06.177 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:27:06.177 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:06.177 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:06.177 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:06.177 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:27:06.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:27:06.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:27:06.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:27:06.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-73729-0 0 2026-02-24T17:27:06.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-73729-0 2026-02-24T17:27:06.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:27:06.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:27:06.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-73729-0 2026-02-24T17:27:06.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:27:06.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:27:06.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-73729-0 td/test-erasure-eio/COPY 2026-02-24T17:27:06.319 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:27:06.320 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:27:06.321 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-24T17:27:06.321 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-24T17:27:06.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-73729-0 td/test-erasure-eio 1 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-73729-0 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:27:06.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:27:06.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-24T17:27:06.325 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:27:06.325 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-73729-0 2026-02-24T17:27:06.325 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:27:06.325 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-0 2026-02-24T17:27:06.325 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-0 2026-02-24T17:27:06.326 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:27:06.540 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:27:06.541 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:06.541 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:27:06.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:27:06.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-24T17:27:06.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:27:06.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-24T17:27:06.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:27:06.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-24T17:27:06.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:27:06.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-24T17:27:06.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:27:06.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-24T17:27:06.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:27:06.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:27:06.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:27:06.544 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-24T17:27:06.544 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:27:06.544 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:27:06.544 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:06.544 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:06.545 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:06.545 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:27:06.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:27:06.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:27:06.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:27:06.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:27:06.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-24T17:27:06.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:27:06.606 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:27:06.606 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:06.607 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:06.607 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:06.607 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:27:06.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:27:06.607 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-73729-0 1 2026-02-24T17:27:06.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-24T17:27:06.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-73729-0 fail 2026-02-24T17:27:06.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:27:06.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:27:06.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-73729-0 2026-02-24T17:27:06.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-24T17:27:06.665 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-24T17:27:06.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-73729-0 td/test-erasure-eio/COPY 2026-02-24T17:27:06.682 INFO:tasks.workunit.client.0.trial066.stderr:error getting pool-jerasure/obj-eio-73729-0: (5) Input/output error 2026-02-24T17:27:06.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-24T17:27:06.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:27:06.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:27:06.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:27:06.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:27:06.946 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:27:06.956 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:27:07.220 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:27:07.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:27:07.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:27:07.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:27:07.230 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:27:07.231 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:27:07.231 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:27:07.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:27:07.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:27:07.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:27:07.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:27:07.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:27:07.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:27:07.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:27:07.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:27:07.344 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:27:07.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:27:07.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:07.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:27:07.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:27:07.346 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:07.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:27:07.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:27:07.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:27:07.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:27:07.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:07.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:07.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:27:07.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:27:07.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:27:07.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:27:07.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:27:07.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:27:07.362 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:27:07.363 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:27:07.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:27:07.365 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:27:07.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:27:07.366 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:27:07.367 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:27:07.368 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:27:07.368 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:27:07.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:07.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:27:07.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:27:07.370 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:07.371 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:27:07.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:27:07.372 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:27:07.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:27:07.373 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:07.374 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:07.374 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:27:07.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:27:07.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:27:07.375 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:27:07.376 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:27:07.376 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:07.377 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:07.377 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:27:07.378 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:27:07.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:27:07.379 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:27:07.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:27:07.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:27:07.401 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:07.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:07.402 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:07.402 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:07.402 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:07.403 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:07.403 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:27:07.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:27:07.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:27:07.439 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:27:07.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:27:07.440 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:07.441 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:27:07.442 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:27:07.442 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:27:07.497 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:27:07.498 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:27:07.498 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:27:07.498 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:27:07.498 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:27:07.499 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:27:07.500 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:27:07.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:27:07.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:27:07.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:27:07.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:27:07.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:27:07.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:27:07.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:27:07.669 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:27:07.669 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:07.669 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:07.669 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:07.669 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:07.669 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:07.669 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:07.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:27:07.671 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:27:07.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:27:07.691 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:27:07.801 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:27:07.810 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:27:08.813 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:27:08.814 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:27:08.860 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:27:08.866 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:27:08.868 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:27:07.433+0000 7fe77345eec0 0 load: jerasure load: lrc 2026-02-24T17:27:08.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-24T17:27:08.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-24T17:27:08.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-24T17:27:08.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-24T17:27:08.869 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:27:08.869 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-24T17:27:08.870 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-24T17:27:08.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:08.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:27:08.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:08.874 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:08.875 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:08.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:27:08.877 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:08.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7a0fbe11-a928-4917-b398-7f8e59d76295 2026-02-24T17:27:08.878 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 7a0fbe11-a928-4917-b398-7f8e59d76295 2026-02-24T17:27:08.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7a0fbe11-a928-4917-b398-7f8e59d76295' 2026-02-24T17:27:08.879 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:08.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBs351pDfFiNRAA3OXLPoLDxHNss+oNncQuPQ== 2026-02-24T17:27:08.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBs351pDfFiNRAA3OXLPoLDxHNss+oNncQuPQ=="}' 2026-02-24T17:27:08.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7a0fbe11-a928-4917-b398-7f8e59d76295 -i td/test-erasure-eio/0/new.json 2026-02-24T17:27:08.997 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:09.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:27:09.006 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBs351pDfFiNRAA3OXLPoLDxHNss+oNncQuPQ== --osd-uuid 7a0fbe11-a928-4917-b398-7f8e59d76295 2026-02-24T17:27:09.022 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.025+0000 7f46870dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:09.023 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.025+0000 7f46870dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:09.024 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.029+0000 7f46870dba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:09.025 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.029+0000 7f46870dba40 -1 bdev(0x5586e1b74400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:09.025 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.029+0000 7f46870dba40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:27:09.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:27:09.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:09.701 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:27:09.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:27:09.701 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:09.815 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:27:09.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:27:09.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:09.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:09.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:09.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:09.873 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.869+0000 7f1f0a4c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:09.877 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.881+0000 7f1f0a4c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:09.884 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:09.885+0000 7f1f0a4c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:09.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:27:09.968 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:09.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:27:09.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:09.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:09.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:09.969 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:09.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:09.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:09.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:27:10.081 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:10.302 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:10.305+0000 7f1f0a4c8a40 -1 Falling back to public interface 2026-02-24T17:27:10.894 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:10.901+0000 7f1f0a4c8a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:27:11.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:11.083 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:11.083 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:11.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:11.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:11.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:27:11.298 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:12.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:12.300 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:12.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:27:12.301 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:27:12.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:12.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:27:12.513 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2856297625,v1:127.0.0.1:6803/2856297625] [v2:127.0.0.1:6804/2856297625,v1:127.0.0.1:6805/2856297625] exists,up 7a0fbe11-a928-4917-b398-7f8e59d76295 2026-02-24T17:27:12.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:12.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:12.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:12.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:12.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:27:12.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:12.517 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:27:12.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:12.519 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:12.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:12.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:12.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:12.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:27:12.522 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:12.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=761cf86c-1433-4689-bb62-d3fa52450318 2026-02-24T17:27:12.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 761cf86c-1433-4689-bb62-d3fa52450318' 2026-02-24T17:27:12.522 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 761cf86c-1433-4689-bb62-d3fa52450318 2026-02-24T17:27:12.522 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:12.530 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBw351pm58XIBAAo7Jqvzvwz5NEEpwmFl+62w== 2026-02-24T17:27:12.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBw351pm58XIBAAo7Jqvzvwz5NEEpwmFl+62w=="}' 2026-02-24T17:27:12.531 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 761cf86c-1433-4689-bb62-d3fa52450318 -i td/test-erasure-eio/1/new.json 2026-02-24T17:27:12.750 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:12.761 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:27:12.762 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBw351pm58XIBAAo7Jqvzvwz5NEEpwmFl+62w== --osd-uuid 761cf86c-1433-4689-bb62-d3fa52450318 2026-02-24T17:27:12.786 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:12.793+0000 7fe36f577a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:12.788 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:12.793+0000 7fe36f577a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:12.789 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:12.797+0000 7fe36f577a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:12.789 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:12.797+0000 7fe36f577a40 -1 bdev(0x563b8d954400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:12.790 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:12.797+0000 7fe36f577a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:27:13.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:27:13.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:13.455 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:27:13.455 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:27:13.456 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:13.754 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:27:13.755 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:27:13.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:13.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:13.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:13.756 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:13.778 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:13.785+0000 7f399ba14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:13.780 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:13.785+0000 7f399ba14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:13.781 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:13.789+0000 7f399ba14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:13.969 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:27:14.174 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:14.181+0000 7f399ba14a40 -1 Falling back to public interface 2026-02-24T17:27:14.184 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:14.751 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:14.757+0000 7f399ba14a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:27:15.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:15.186 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:15.187 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:15.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:15.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:15.188 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:27:15.394 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1285427488,v1:127.0.0.1:6811/1285427488] [v2:127.0.0.1:6812/1285427488,v1:127.0.0.1:6813/1285427488] exists,up 761cf86c-1433-4689-bb62-d3fa52450318 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:15.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:15.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:15.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:27:15.398 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:15.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3398f716-86ed-42cc-a115-5d7f24fbb9b3 2026-02-24T17:27:15.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3398f716-86ed-42cc-a115-5d7f24fbb9b3' 2026-02-24T17:27:15.399 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 3398f716-86ed-42cc-a115-5d7f24fbb9b3 2026-02-24T17:27:15.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:15.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBz351phAT5GBAAphMgqeUKlN/YEBipnuQA6Q== 2026-02-24T17:27:15.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBz351phAT5GBAAphMgqeUKlN/YEBipnuQA6Q=="}' 2026-02-24T17:27:15.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3398f716-86ed-42cc-a115-5d7f24fbb9b3 -i td/test-erasure-eio/2/new.json 2026-02-24T17:27:15.639 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:27:15.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:27:15.649 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBz351phAT5GBAAphMgqeUKlN/YEBipnuQA6Q== --osd-uuid 3398f716-86ed-42cc-a115-5d7f24fbb9b3 2026-02-24T17:27:15.666 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:15.673+0000 7f0644c8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:15.668 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:15.673+0000 7f0644c8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:15.669 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:15.677+0000 7f0644c8fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:15.669 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:15.677+0000 7f0644c8fa40 -1 bdev(0x559e2ba70400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:15.669 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:15.677+0000 7f0644c8fa40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:27:16.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:27:16.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:16.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:27:16.384 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:27:16.385 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:16.683 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:27:16.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:27:16.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:16.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:16.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:16.685 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:16.699 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:16.704+0000 7ff3e6efea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:16.701 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:16.708+0000 7ff3e6efea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:16.702 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:16.708+0000 7ff3e6efea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:16.899 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:16.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:16.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:27:17.106 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:17.112+0000 7ff3e6efea40 -1 Falling back to public interface 2026-02-24T17:27:17.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:17.671 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:17.676+0000 7ff3e6efea40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:27:18.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:18.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:18.123 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:18.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:18.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:18.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:27:18.333 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3199072261,v1:127.0.0.1:6819/3199072261] [v2:127.0.0.1:6820/3199072261,v1:127.0.0.1:6821/3199072261] exists,up 3398f716-86ed-42cc-a115-5d7f24fbb9b3 2026-02-24T17:27:18.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:18.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:18.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:18.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:18.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:27:18.338 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:18.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:18.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:27:18.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:18.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=96c5d28e-fb66-4ca3-b0f8-5e225c1d0dd4 2026-02-24T17:27:18.341 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 96c5d28e-fb66-4ca3-b0f8-5e225c1d0dd4 2026-02-24T17:27:18.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 96c5d28e-fb66-4ca3-b0f8-5e225c1d0dd4' 2026-02-24T17:27:18.342 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:18.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB2351pY1FeFRAAMHnQ4OfSwyGISrR8RwzLkg== 2026-02-24T17:27:18.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB2351pY1FeFRAAMHnQ4OfSwyGISrR8RwzLkg=="}' 2026-02-24T17:27:18.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 96c5d28e-fb66-4ca3-b0f8-5e225c1d0dd4 -i td/test-erasure-eio/3/new.json 2026-02-24T17:27:18.577 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:27:18.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:27:18.588 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB2351pY1FeFRAAMHnQ4OfSwyGISrR8RwzLkg== --osd-uuid 96c5d28e-fb66-4ca3-b0f8-5e225c1d0dd4 2026-02-24T17:27:18.605 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:18.612+0000 7f98a1823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:18.607 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:18.612+0000 7f98a1823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:18.608 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:18.612+0000 7f98a1823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:18.608 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:18.616+0000 7f98a1823a40 -1 bdev(0x55e249fe8400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:18.608 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:18.616+0000 7f98a1823a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:27:19.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:27:19.260 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:19.261 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:27:19.261 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:27:19.262 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:19.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:27:19.563 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:27:19.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:19.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:19.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:19.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:19.588 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:19.592+0000 7fa3ec39ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:19.589 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:19.596+0000 7fa3ec39ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:19.590 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:19.596+0000 7fa3ec39ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:19.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:27:19.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:19.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:27:19.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:19.772 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:19.773 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:19.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:19.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:19.773 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:19.774 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:27:19.922 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:19.928+0000 7fa3ec39ca40 -1 Falling back to public interface 2026-02-24T17:27:19.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:20.582 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:20.588+0000 7fa3ec39ca40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:27:20.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:20.990 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:20.991 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:20.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:20.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:20.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:27:21.218 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3538299765,v1:127.0.0.1:6827/3538299765] [v2:127.0.0.1:6828/3538299765,v1:127.0.0.1:6829/3538299765] exists,up 96c5d28e-fb66-4ca3-b0f8-5e225c1d0dd4 2026-02-24T17:27:21.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:21.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:21.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:21.220 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:27:21.220 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:27:21.221 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:27:21.221 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:21.221 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:21.221 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:21.221 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:27:21.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:27:21.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:27:21.281 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:27:21.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:27:10.309+0000 7f1f0a4c8a40 0 load: jerasure load: lrc 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:27:21.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-24T17:27:21.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:27:21.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:27:21.965 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:27:21.975 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:27:22.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:27:22.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:27:22.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:27:22.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:27:22.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:27:22.978 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:27:22.979 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:27:22.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:27:22.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:27:22.979 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:27:23.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:27:23.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:27:23.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:27:23.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:27:23.089 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:27:23.089 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:27:23.301 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:27:23.302 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:23.302 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:27:23.302 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:27:23.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:27:23.302 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:23.302 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:27:23.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-24T17:27:23.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-24T17:27:23.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-24T17:27:23.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:23.382 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:27:23.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-24T17:27:23.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-24T17:27:23.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-24T17:27:23.459 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:23.460 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:27:23.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=81604378627 2026-02-24T17:27:23.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 81604378627 2026-02-24T17:27:23.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378627' 2026-02-24T17:27:23.538 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:23.538 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:27:23.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149699 2026-02-24T17:27:23.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149699 2026-02-24T17:27:23.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-81604378627 3-111669149699' 2026-02-24T17:27:23.618 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:23.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-24T17:27:23.619 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:23.620 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:27:23.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-24T17:27:23.620 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:23.621 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-24T17:27:23.621 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803781 2026-02-24T17:27:23.622 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-24T17:27:23.622 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:23.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803778 -lt 25769803781 2026-02-24T17:27:23.834 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:27:24.835 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:27:24.836 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:25.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-24T17:27:25.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:25.054 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574852 2026-02-24T17:27:25.054 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:25.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:27:25.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574852 2026-02-24T17:27:25.056 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:25.057 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-24T17:27:25.057 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 55834574852 2026-02-24T17:27:25.058 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-24T17:27:25.058 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:27:25.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-24T17:27:25.275 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:25.276 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-81604378627 2026-02-24T17:27:25.276 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:25.277 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:27:25.278 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-81604378627 2026-02-24T17:27:25.278 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:25.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=81604378627 2026-02-24T17:27:25.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 81604378627' 2026-02-24T17:27:25.280 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 81604378627 2026-02-24T17:27:25.280 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:27:25.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 81604378628 -lt 81604378627 2026-02-24T17:27:25.504 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:25.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149699 2026-02-24T17:27:25.505 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:25.506 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:27:25.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149699 2026-02-24T17:27:25.507 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:25.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149699 2026-02-24T17:27:25.510 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-24T17:27:25.510 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 111669149699 2026-02-24T17:27:25.510 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:27:25.722 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-24T17:27:25.722 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:27:25.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:25.723 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:26.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:27:26.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:27:26.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:27:26.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:27:26.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:27:26.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:27:26.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:27:26.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:27:26.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:27:26.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:27:26.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:26.208 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:26.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:27:26.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:27:26.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:27:26.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-24T17:27:26.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-73729-1 2026-02-24T17:27:26.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-73729-1 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-73729-1 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:27:26.496 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:26.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:27:26.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-73729-1 td/test-erasure-eio/ORIGINAL 2026-02-24T17:27:26.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-73729-1 td/test-erasure-eio 1 2026-02-24T17:27:26.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:27:26.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-73729-1 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-24T17:27:26.514 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:27:26.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-73729-1 2026-02-24T17:27:26.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:27:26.515 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-1 2026-02-24T17:27:26.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-1 2026-02-24T17:27:26.515 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:27:26.729 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:27:26.730 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:26.730 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:27:26.730 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:27:26.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-24T17:27:26.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:27:26.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-24T17:27:26.730 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:27:26.731 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-24T17:27:26.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:27:26.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-24T17:27:26.731 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:27:26.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-24T17:27:26.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:27:26.732 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:27:26.732 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:27:26.733 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-24T17:27:26.733 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:27:26.733 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:27:26.734 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:26.734 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:26.734 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:26.734 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:27:26.734 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:27:26.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:27:26.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:27:26.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:27:26.798 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-24T17:27:26.798 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:27:26.798 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:27:26.798 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:26.799 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:26.799 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:26.799 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:27:26.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:27:26.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-73729-1 1 2026-02-24T17:27:26.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-73729-1 2026-02-24T17:27:26.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:27:26.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:27:26.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-73729-1 2026-02-24T17:27:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:27:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:27:26.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-73729-1 td/test-erasure-eio/COPY 2026-02-24T17:27:26.879 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:27:26.880 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:27:26.881 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-24T17:27:26.881 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-24T17:27:26.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-24T17:27:26.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-73729-1 td/test-erasure-eio 2 2026-02-24T17:27:26.883 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-73729-1 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=2 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:27:26.884 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-73729-1 2026-02-24T17:27:26.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:27:26.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-1 2026-02-24T17:27:26.885 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-1 2026-02-24T17:27:26.885 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:27:27.099 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:27:27.099 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:27.100 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:27:27.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:27:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-24T17:27:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:27:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=2 2026-02-24T17:27:27.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:27:27.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/2/type 2026-02-24T17:27:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:27:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-24T17:27:27.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:27.103 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:27.104 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:27.104 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.2.asok 2026-02-24T17:27:27.104 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:27:27.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:27:27.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:27:27.164 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:27:27.164 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.2 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.2.asok 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:27:27.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-73729-1 2 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-73729-1 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-73729-1 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:27:27.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-73729-1 td/test-erasure-eio/COPY 2026-02-24T17:27:27.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:27:27.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:27:27.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:27:27.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:27:27.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:27:27.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:27:27.510 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:27:27.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:27:27.798 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:27:27.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:27:27.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:27:27.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:27:27.807 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:27:27.807 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:27:27.807 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:27:27.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:27:27.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:27:27.808 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:27:27.918 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:27:27.918 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:27:27.919 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:27:27.919 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:27:27.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:27:27.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:27:27.921 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:27:27.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:27.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:27:27.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:27:27.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:27.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:27:27.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:27:27.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:27:27.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:27:27.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:27.936 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:27.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:27:27.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:27:27.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:27:27.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:27:27.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:27:27.939 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:27:27.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:27:27.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:27:27.941 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:27:27.942 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:27:27.942 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:27:27.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:27:27.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:27:27.944 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:27:27.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:27.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:27:27.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:27:27.946 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:27:27.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:27:27.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:27:27.948 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:27:27.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:27:27.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:27.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:27.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:27:27.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:27:27.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:27:27.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:27:27.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:27:27.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:27.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:27.952 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:27:27.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:27:27.955 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:27:27.976 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:27:27.976 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:27.976 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:27.977 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:27.977 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:27.977 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:27.977 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:27.977 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:27:28.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:27:28.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:27:28.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:27:28.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:27:28.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:27:28.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:27:28.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:27:28.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:27:28.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:27:28.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:27:28.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:27:28.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:27:28.067 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:27:28.068 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:27:28.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:27:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:27:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:27:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:27:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:27:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:27:28.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:27:28.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:27:28.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:28.234 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:28.234 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:28.235 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:28.235 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:28.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:28.235 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:27:28.236 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:27:28.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:27:28.254 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:27:28.373 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:27:28.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:27:29.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:27:29.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:27:29.384 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:27:29.384 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:29.384 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:29.384 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:29.384 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:27:29.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:27:29.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:27:29.432 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:27:29.438 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:27:29.440 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:27:28.004+0000 7f22b5d47ec0 0 load: jerasure load: lrc 2026-02-24T17:27:29.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-24T17:27:29.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-24T17:27:29.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-24T17:27:29.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-24T17:27:29.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:27:29.441 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-24T17:27:29.442 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:29.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:29.444 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:29.444 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:29.444 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:29.444 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:29.444 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:29.444 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:29.444 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:29.445 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:27:29.446 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:29.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=84723811-b17c-4596-90ca-18a05a91f10e 2026-02-24T17:27:29.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 84723811-b17c-4596-90ca-18a05a91f10e' 2026-02-24T17:27:29.447 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 84723811-b17c-4596-90ca-18a05a91f10e 2026-02-24T17:27:29.447 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:29.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCB351pd5TCGxAAsymiOAIionQYf5iuow3CEg== 2026-02-24T17:27:29.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCB351pd5TCGxAAsymiOAIionQYf5iuow3CEg=="}' 2026-02-24T17:27:29.458 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 84723811-b17c-4596-90ca-18a05a91f10e -i td/test-erasure-eio/0/new.json 2026-02-24T17:27:29.571 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:29.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:27:29.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCB351pd5TCGxAAsymiOAIionQYf5iuow3CEg== --osd-uuid 84723811-b17c-4596-90ca-18a05a91f10e 2026-02-24T17:27:29.595 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:29.600+0000 7f8167feca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:29.597 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:29.600+0000 7f8167feca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:29.598 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:29.604+0000 7f8167feca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:29.599 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:29.604+0000 7f8167feca40 -1 bdev(0x55be86052400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:29.599 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:29.604+0000 7f8167feca40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:27:30.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:27:30.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:30.273 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:27:30.274 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:27:30.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:30.389 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:27:30.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:27:30.390 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:30.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:30.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:30.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:30.455 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:30.456+0000 7fc1d1b2ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:30.461 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:30.464+0000 7fc1d1b2ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:30.461 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:30.468+0000 7fc1d1b2ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:30.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:27:30.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:30.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:27:30.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:30.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:30.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:30.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:30.547 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:30.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:30.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:27:30.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:30.839 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:30.844+0000 7fc1d1b2ca40 -1 Falling back to public interface 2026-02-24T17:27:31.408 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:31.412+0000 7fc1d1b2ca40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:27:31.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:31.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:31.663 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:31.663 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:31.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:31.664 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:27:31.876 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:32.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:32.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:32.877 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:27:32.877 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:27:32.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:32.878 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:27:33.093 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/821312549,v1:127.0.0.1:6803/821312549] [v2:127.0.0.1:6804/821312549,v1:127.0.0.1:6805/821312549] exists,up 84723811-b17c-4596-90ca-18a05a91f10e 2026-02-24T17:27:33.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:33.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:33.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:33.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:33.093 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:33.094 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:33.095 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:27:33.097 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:33.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1d9dec3c-4f0d-4764-9143-a015cb1e710d 2026-02-24T17:27:33.098 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 1d9dec3c-4f0d-4764-9143-a015cb1e710d 2026-02-24T17:27:33.098 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1d9dec3c-4f0d-4764-9143-a015cb1e710d' 2026-02-24T17:27:33.098 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:33.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCF351pb6ALBxAAyNo7q/49R+hLbZMnYt5qRA== 2026-02-24T17:27:33.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCF351pb6ALBxAAyNo7q/49R+hLbZMnYt5qRA=="}' 2026-02-24T17:27:33.111 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1d9dec3c-4f0d-4764-9143-a015cb1e710d -i td/test-erasure-eio/1/new.json 2026-02-24T17:27:33.340 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:33.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:27:33.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCF351pb6ALBxAAyNo7q/49R+hLbZMnYt5qRA== --osd-uuid 1d9dec3c-4f0d-4764-9143-a015cb1e710d 2026-02-24T17:27:33.367 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:33.371+0000 7fb3855a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:33.368 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:33.371+0000 7fb3855a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:33.369 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:33.375+0000 7fb3855a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:33.370 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:33.375+0000 7fb3855a7a40 -1 bdev(0x55c919d68400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:33.370 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:33.375+0000 7fb3855a7a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:27:33.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:27:33.992 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:33.993 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:27:33.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:27:33.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:34.285 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:27:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:27:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:34.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:34.302 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:34.307+0000 7f9a7d9c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:34.304 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:34.307+0000 7f9a7d9c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:34.305 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:34.307+0000 7f9a7d9c2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:34.493 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:27:34.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:34.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:27:34.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:34.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:34.494 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:34.494 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:34.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:34.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:34.495 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:27:34.723 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:34.735 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:34.739+0000 7f9a7d9c2a40 -1 Falling back to public interface 2026-02-24T17:27:35.398 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:35.403+0000 7f9a7d9c2a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:27:35.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:35.724 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:35.725 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:35.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:35.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:35.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:27:35.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:36.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:36.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:36.945 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:27:36.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:27:36.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:36.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:27:37.164 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1417159925,v1:127.0.0.1:6811/1417159925] [v2:127.0.0.1:6812/1417159925,v1:127.0.0.1:6813/1417159925] exists,up 1d9dec3c-4f0d-4764-9143-a015cb1e710d 2026-02-24T17:27:37.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:37.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:37.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:37.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:27:37.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:37.169 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:37.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:27:37.171 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:37.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bfed4715-057f-42f5-b7da-ba6ac6b3586e 2026-02-24T17:27:37.171 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 bfed4715-057f-42f5-b7da-ba6ac6b3586e 2026-02-24T17:27:37.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 bfed4715-057f-42f5-b7da-ba6ac6b3586e' 2026-02-24T17:27:37.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:37.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJ351p8jIrCxAAjlhLLJw447NpzrogMgh8lA== 2026-02-24T17:27:37.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJ351p8jIrCxAAjlhLLJw447NpzrogMgh8lA=="}' 2026-02-24T17:27:37.181 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bfed4715-057f-42f5-b7da-ba6ac6b3586e -i td/test-erasure-eio/2/new.json 2026-02-24T17:27:37.404 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:27:37.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:27:37.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJ351p8jIrCxAAjlhLLJw447NpzrogMgh8lA== --osd-uuid bfed4715-057f-42f5-b7da-ba6ac6b3586e 2026-02-24T17:27:37.431 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:37.435+0000 7ffaaa98fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:37.432 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:37.435+0000 7ffaaa98fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:37.433 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:37.439+0000 7ffaaa98fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:37.434 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:37.439+0000 7ffaaa98fa40 -1 bdev(0x55618ef52400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:37.434 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:37.439+0000 7ffaaa98fa40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:27:38.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:27:38.060 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:38.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:27:38.061 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:27:38.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:38.349 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:27:38.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:27:38.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:38.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:38.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:38.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:38.373 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:38.375+0000 7f8496931a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:38.375 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:38.379+0000 7f8496931a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:38.376 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:38.379+0000 7f8496931a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:38.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:27:38.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:38.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:27:38.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:38.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:38.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:38.564 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:38.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:38.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:38.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:27:38.775 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:38.791 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:38.795+0000 7f8496931a40 -1 Falling back to public interface 2026-02-24T17:27:39.363 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:39.367+0000 7f8496931a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:27:39.776 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:39.777 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:39.777 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:39.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:39.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:39.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:27:39.994 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2873039762,v1:127.0.0.1:6819/2873039762] [v2:127.0.0.1:6820/2873039762,v1:127.0.0.1:6821/2873039762] exists,up bfed4715-057f-42f5-b7da-ba6ac6b3586e 2026-02-24T17:27:39.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:39.997 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:39.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:39.999 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:27:40.000 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:27:40.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:27:40.001 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:27:40.001 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=83cbd0a6-b1b2-43ce-9d3d-4f985c38060c 2026-02-24T17:27:40.002 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 83cbd0a6-b1b2-43ce-9d3d-4f985c38060c 2026-02-24T17:27:40.002 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 83cbd0a6-b1b2-43ce-9d3d-4f985c38060c' 2026-02-24T17:27:40.002 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:27:40.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCM351pUar9ABAAP7NyCo+KKb4ArbnjvBxWjg== 2026-02-24T17:27:40.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCM351pUar9ABAAP7NyCo+KKb4ArbnjvBxWjg=="}' 2026-02-24T17:27:40.010 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 83cbd0a6-b1b2-43ce-9d3d-4f985c38060c -i td/test-erasure-eio/3/new.json 2026-02-24T17:27:40.222 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:27:40.232 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:27:40.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCM351pUar9ABAAP7NyCo+KKb4ArbnjvBxWjg== --osd-uuid 83cbd0a6-b1b2-43ce-9d3d-4f985c38060c 2026-02-24T17:27:40.250 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:40.255+0000 7fa7c0aaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:40.251 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:40.255+0000 7fa7c0aaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:40.252 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:40.255+0000 7fa7c0aaea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:40.253 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:40.255+0000 7fa7c0aaea40 -1 bdev(0x558c17678400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:27:40.253 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:40.255+0000 7fa7c0aaea40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:27:40.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:27:40.893 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:27:40.894 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:27:40.894 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:27:40.895 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:27:41.191 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:27:41.192 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:27:41.192 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:41.192 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:27:41.192 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:27:41.192 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:27:41.215 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:41.219+0000 7f37c716ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:41.216 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:41.219+0000 7f37c716ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:41.217 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:41.219+0000 7f37c716ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:41.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:27:41.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:41.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:27:41.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:41.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:41.412 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:27:41.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:41.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:41.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:41.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:27:41.623 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:41.627+0000 7f37c716ba40 -1 Falling back to public interface 2026-02-24T17:27:41.624 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:42.246 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:42.251+0000 7f37c716ba40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:27:42.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:42.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:42.626 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:27:42.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:42.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:42.627 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:27:42.841 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:43.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:43.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:43.843 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:27:43.844 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:27:43.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:43.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:27:44.052 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 26 up_thru 29 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3417976001,v1:127.0.0.1:6827/3417976001] [v2:127.0.0.1:6828/3417976001,v1:127.0.0.1:6829/3417976001] exists,up 83cbd0a6-b1b2-43ce-9d3d-4f985c38060c 2026-02-24T17:27:44.053 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:44.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:44.054 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:44.054 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:27:44.054 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:27:44.054 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:27:44.055 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:27:44.055 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:44.055 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:44.055 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:27:44.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:27:44.055 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:27:44.116 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:27:44.122 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:27:44.124 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:27:30.848+0000 7fc1d1b2ca40 0 load: jerasure load: lrc 2026-02-24T17:27:44.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-24T17:27:44.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-24T17:27:44.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:27:44.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:27:44.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-24T17:27:44.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:27:44.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-24T17:27:44.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:27:44.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-24T17:27:44.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:27:44.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:27:44.791 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:27:44.802 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:27:45.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:27:45.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:27:45.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:27:45.805 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:27:45.806 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:27:45.806 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:27:45.806 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:27:45.807 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:27:45.807 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:27:45.807 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:27:45.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:27:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:27:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:27:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:27:45.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:27:45.905 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:27:46.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:27:46.128 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:46.128 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:27:46.128 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:27:46.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:27:46.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:46.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:27:46.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-24T17:27:46.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-24T17:27:46.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-24T17:27:46.207 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:46.207 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:27:46.287 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607556 2026-02-24T17:27:46.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607556 2026-02-24T17:27:46.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-24T17:27:46.288 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:46.288 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:27:46.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411332 2026-02-24T17:27:46.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411332 2026-02-24T17:27:46.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411332' 2026-02-24T17:27:46.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:46.366 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:27:46.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149699 2026-02-24T17:27:46.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149699 2026-02-24T17:27:46.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-77309411332 3-111669149699' 2026-02-24T17:27:46.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:46.448 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-24T17:27:46.448 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:46.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:27:46.449 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-24T17:27:46.449 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:46.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-24T17:27:46.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-24T17:27:46.451 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803781 2026-02-24T17:27:46.451 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:46.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-24T17:27:46.672 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:46.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607556 2026-02-24T17:27:46.673 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:46.674 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:27:46.675 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607556 2026-02-24T17:27:46.675 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:46.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607556 2026-02-24T17:27:46.677 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-24T17:27:46.678 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607556 2026-02-24T17:27:46.678 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:27:46.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-24T17:27:46.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:46.902 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411332 2026-02-24T17:27:46.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:46.903 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:27:46.904 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411332 2026-02-24T17:27:46.904 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:46.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411332 2026-02-24T17:27:46.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411332' 2026-02-24T17:27:46.906 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411332 2026-02-24T17:27:46.907 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:27:47.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411332 -lt 77309411332 2026-02-24T17:27:47.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:47.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149699 2026-02-24T17:27:47.123 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:47.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:27:47.125 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149699 2026-02-24T17:27:47.125 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:47.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149699 2026-02-24T17:27:47.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-24T17:27:47.127 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 111669149699 2026-02-24T17:27:47.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:27:47.347 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-24T17:27:47.348 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:27:47.348 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:47.349 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:47.625 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:27:47.626 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:27:47.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:27:47.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:27:47.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:27:47.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:27:47.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:27:47.626 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:27:47.830 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:27:47.830 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:27:47.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:47.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:48.109 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-73729-1 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-73729-1 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-73729-1 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:27:48.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-73729-1 td/test-erasure-eio/ORIGINAL 2026-02-24T17:27:48.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-73729-1 td/test-erasure-eio 1 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-73729-1 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-73729-1 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:27:48.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-73729-1 2026-02-24T17:27:48.131 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-73729-1 2026-02-24T17:27:48.131 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:27:48.349 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:27:48.349 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:48.350 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:27:48.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:27:48.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-73729-1 remove 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-73729-1 remove 2026-02-24T17:27:48.352 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:27:48.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:27:48.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:27:48.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-73729-1 remove 2026-02-24T17:27:48.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:27:48.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:27:48.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-24T17:27:48.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:27:48.661 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-24T17:27:48.662 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-73729-1 remove 2026-02-24T17:27:48.851 INFO:tasks.workunit.client.0.trial066.stdout:remove 1#2:d5347817:::obj-remove-73729-1:head# 2026-02-24T17:27:48.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-24T17:27:48.944 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:27:48.945 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:48.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:27:48.947 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr:start osd.1 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:27:48.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:27:48.965 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:48.967+0000 7f90ae00ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:48.967 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:48.971+0000 7f90ae00ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:48.967 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:48.971+0000 7f90ae00ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:49.163 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:27:49.164 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:49.164 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:27:49.303 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:49.307+0000 7f90ae00ca40 -1 Falling back to public interface 2026-02-24T17:27:49.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:50.053 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:50.055+0000 7f90ae00ca40 -1 osd.1 35 log_to_monitors true 2026-02-24T17:27:50.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:50.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:50.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:50.383 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:50.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:50.384 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:27:50.603 INFO:tasks.workunit.client.0.trial066.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [12,35) [v2:127.0.0.1:6810/1986586256,v1:127.0.0.1:6811/1986586256] [v2:127.0.0.1:6812/1986586256,v1:127.0.0.1:6813/1986586256] exists,up 1d9dec3c-4f0d-4764-9143-a015cb1e710d 2026-02-24T17:27:50.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:50.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:50.603 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:50.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:27:50.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:27:50.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:27:50.604 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:27:50.604 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:27:50.605 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:27:50.605 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:27:50.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:27:50.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:27:50.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:27:50.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:27:50.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:27:50.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:27:50.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:27:50.703 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:27:50.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:27:50.921 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:27:50.921 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:50.921 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:27:50.922 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:27:50.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:27:50.922 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:50.922 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:27:51.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803784 2026-02-24T17:27:51.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803784 2026-02-24T17:27:51.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784' 2026-02-24T17:27:51.005 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:51.005 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:27:51.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724547 2026-02-24T17:27:51.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724547 2026-02-24T17:27:51.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-167503724547' 2026-02-24T17:27:51.084 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:51.084 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:27:51.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411335 2026-02-24T17:27:51.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411335 2026-02-24T17:27:51.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335' 2026-02-24T17:27:51.164 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:51.164 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:27:51.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149702 2026-02-24T17:27:51.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149702 2026-02-24T17:27:51.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335 3-111669149702' 2026-02-24T17:27:51.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:51.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803784 2026-02-24T17:27:51.244 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:51.245 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:27:51.246 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803784 2026-02-24T17:27:51.246 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:51.247 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803784 2026-02-24T17:27:51.247 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-24T17:27:51.247 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803784 2026-02-24T17:27:51.248 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:51.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-24T17:27:51.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:27:52.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:27:52.469 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:52.686 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-24T17:27:52.686 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:52.686 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-167503724547 2026-02-24T17:27:52.686 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:52.688 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:27:52.688 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-167503724547 2026-02-24T17:27:52.689 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:52.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724547 2026-02-24T17:27:52.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 167503724547' 2026-02-24T17:27:52.690 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 167503724547 2026-02-24T17:27:52.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:27:52.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-24T17:27:52.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:52.905 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411335 2026-02-24T17:27:52.905 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:52.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:27:52.907 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411335 2026-02-24T17:27:52.907 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:52.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411335 2026-02-24T17:27:52.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-24T17:27:52.909 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 77309411335 2026-02-24T17:27:52.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:27:53.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-24T17:27:53.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:53.122 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149702 2026-02-24T17:27:53.122 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:53.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:27:53.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149702 2026-02-24T17:27:53.124 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:53.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149702 2026-02-24T17:27:53.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-24T17:27:53.126 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 111669149702 2026-02-24T17:27:53.126 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:27:53.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-24T17:27:53.359 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:27:53.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:53.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:53.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:27:53.647 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:27:53.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:27:53.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:27:53.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:27:53.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:27:53.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:27:53.648 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:27:53.861 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:27:53.861 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:27:53.862 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:53.862 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-73729-1 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-73729-1 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:27:54.147 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-73729-1 td/test-erasure-eio/COPY 2026-02-24T17:27:54.166 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:27:54.167 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:27:54.168 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-24T17:27:54.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-24T17:27:54.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-24T17:27:54.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-73729-1 td/test-erasure-eio 2 2026-02-24T17:27:54.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-24T17:27:54.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-24T17:27:54.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-24T17:27:54.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-24T17:27:54.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-24T17:27:54.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-24T17:27:54.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-73729-1 2026-02-24T17:27:54.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-73729-1 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-73729-1 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-73729-1 2026-02-24T17:27:54.174 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:27:54.390 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:27:54.390 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:54.390 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:27:54.390 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-73729-1 remove 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-73729-1 remove 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-24T17:27:54.391 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:27:54.392 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:27:54.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:27:54.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-73729-1 remove 2026-02-24T17:27:54.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-24T17:27:54.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-24T17:27:54.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-24T17:27:54.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-24T17:27:54.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-24T17:27:54.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-73729-1 remove 2026-02-24T17:27:54.906 INFO:tasks.workunit.client.0.trial066.stdout:remove 2#2:d5347817:::obj-remove-73729-1:head# 2026-02-24T17:27:55.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-24T17:27:55.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:27:55.019 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:27:55.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:27:55.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:27:55.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:27:55.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:27:55.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:27:55.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:27:55.020 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:27:55.021 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:27:55.022 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:27:55.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:27:55.024 INFO:tasks.workunit.client.0.trial066.stderr:start osd.2 2026-02-24T17:27:55.024 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:27:55.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:27:55.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:27:55.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:27:55.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:27:55.025 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:27:55.036 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:55.038+0000 7f69f5a78a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:55.037 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:55.038+0000 7f69f5a78a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:55.038 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:55.038+0000 7f69f5a78a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:0 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:55.231 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:27:55.428 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:55.430+0000 7f69f5a78a40 -1 Falling back to public interface 2026-02-24T17:27:55.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:27:56.069 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:27:56.070+0000 7f69f5a78a40 -1 osd.2 40 log_to_monitors true 2026-02-24T17:27:56.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:27:56.453 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:27:56.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:27:56.454 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:56.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:27:56.454 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 up in weight 1 up_from 44 up_thru 36 down_at 41 last_clean_interval [18,40) [v2:127.0.0.1:6818/4041931939,v1:127.0.0.1:6819/4041931939] [v2:127.0.0.1:6820/4041931939,v1:127.0.0.1:6821/4041931939] exists,up bfed4715-057f-42f5-b7da-ba6ac6b3586e 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:27:56.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:27:56.671 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:27:56.671 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:27:56.671 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:27:56.672 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:27:56.672 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:27:56.672 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:27:56.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:27:56.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:27:56.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:27:56.778 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:27:56.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:27:56.779 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:27:56.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:27:56.989 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:27:56.989 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:27:56.989 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:27:56.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:27:56.989 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:56.990 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:27:57.069 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803787 2026-02-24T17:27:57.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803787 2026-02-24T17:27:57.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787' 2026-02-24T17:27:57.070 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:57.070 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:27:57.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=167503724550 2026-02-24T17:27:57.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 167503724550 2026-02-24T17:27:57.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-167503724550' 2026-02-24T17:27:57.149 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:57.149 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:27:57.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=188978561027 2026-02-24T17:27:57.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 188978561027 2026-02-24T17:27:57.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-188978561027' 2026-02-24T17:27:57.227 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:27:57.227 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:27:57.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149705 2026-02-24T17:27:57.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149705 2026-02-24T17:27:57.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-188978561027 3-111669149705' 2026-02-24T17:27:57.305 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:57.306 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803787 2026-02-24T17:27:57.306 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:57.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:27:57.307 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803787 2026-02-24T17:27:57.308 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:57.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803787 2026-02-24T17:27:57.309 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-24T17:27:57.309 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.0 seq 25769803787 2026-02-24T17:27:57.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:57.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-24T17:27:57.518 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:27:58.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:27:58.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:27:58.740 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803788 -lt 25769803787 2026-02-24T17:27:58.741 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:58.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-167503724550 2026-02-24T17:27:58.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:58.742 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:27:58.743 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-167503724550 2026-02-24T17:27:58.743 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:58.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=167503724550 2026-02-24T17:27:58.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 167503724550' 2026-02-24T17:27:58.745 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.1 seq 167503724550 2026-02-24T17:27:58.745 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:27:58.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-24T17:27:58.957 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:58.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-188978561027 2026-02-24T17:27:58.958 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:58.959 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:27:58.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-188978561027 2026-02-24T17:27:58.960 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:58.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=188978561027 2026-02-24T17:27:58.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 188978561027' 2026-02-24T17:27:58.962 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.2 seq 188978561027 2026-02-24T17:27:58.962 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:27:59.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 188978561027 -lt 188978561027 2026-02-24T17:27:59.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:27:59.190 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149705 2026-02-24T17:27:59.190 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:27:59.191 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:27:59.191 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149705 2026-02-24T17:27:59.192 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:27:59.193 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149705 2026-02-24T17:27:59.193 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-24T17:27:59.193 INFO:tasks.workunit.client.0.trial066.stderr:waiting osd.3 seq 111669149705 2026-02-24T17:27:59.194 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:27:59.414 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149706 -lt 111669149705 2026-02-24T17:27:59.415 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:27:59.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:59.416 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:27:59.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:27:59.690 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:27:59.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:27:59.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:27:59.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:27:59.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:27:59.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:27:59.691 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:27:59.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:27:59.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:27:59.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:27:59.903 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:00.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:28:00.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:28:00.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:28:00.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-24T17:28:00.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-73729-1 2026-02-24T17:28:00.195 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:28:00.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:28:00.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-73729-1 2026-02-24T17:28:00.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:28:00.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:28:00.196 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-73729-1 td/test-erasure-eio/COPY 2026-02-24T17:28:00.213 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:28:00.214 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:28:00.215 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-24T17:28:00.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:28:00.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:28:00.217 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:28:00.499 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:28:00.509 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:28:00.775 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:28:00.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:28:00.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:28:00.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:28:00.786 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:28:00.787 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:00.787 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:00.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:00.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:00.787 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:00.897 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:00.898 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:28:00.898 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:28:00.899 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:28:00.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:28:00.900 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:28:00.900 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:28:00.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:00.901 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:28:00.902 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:28:00.902 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:00.903 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:28:00.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:28:00.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:28:00.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:28:00.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:00.920 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:00.920 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:00.923 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:00.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:00.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:00.924 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:00.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:00.926 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:28:00.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:28:00.927 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:28:00.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:28:00.928 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:28:00.929 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:28:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:28:00.930 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:28:00.931 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:00.932 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:28:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:28:00.933 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:28:00.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:28:00.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:00.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:00.935 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:28:00.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:28:00.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:28:00.936 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:28:00.937 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:28:00.938 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:00.938 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:00.938 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:28:00.939 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:28:00.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:28:00.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:28:00.940 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:28:00.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:28:00.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:28:00.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:28:00.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:28:00.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:28:00.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:28:00.941 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:28:00.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:28:00.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:00.964 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:00.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:00.964 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:00.965 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:00.965 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:00.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:28:00.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:28:00.993 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:28:00.993 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:28:00.993 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:28:00.993 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:28:00.993 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:28:00.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:28:01.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:28:01.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:28:01.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:28:01.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:28:01.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:28:01.049 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:28:01.049 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:28:01.050 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:28:01.050 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:01.050 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:01.050 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:01.050 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:28:01.051 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:28:01.051 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:28:01.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:28:01.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:28:01.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:28:01.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:28:01.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:28:01.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:28:01.105 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:28:01.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:28:01.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:01.215 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:01.216 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:01.216 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:01.216 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:01.216 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:01.217 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:28:01.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:28:01.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:28:01.238 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:28:01.358 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:28:01.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:28:02.368 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:28:02.369 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:28:02.417 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:28:02.423 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:28:02.425 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:28:00.982+0000 7fd12dc2fec0 0 load: jerasure load: lrc 2026-02-24T17:28:02.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-24T17:28:02.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-24T17:28:02.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-24T17:28:02.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-24T17:28:02.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-24T17:28:02.426 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:28:02.427 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-24T17:28:02.427 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-24T17:28:02.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:02.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:28:02.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:02.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:02.431 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:02.432 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:02.433 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:28:02.434 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:02.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0cc3dbc1-48a8-421e-b65d-21d9823bb3aa 2026-02-24T17:28:02.435 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0cc3dbc1-48a8-421e-b65d-21d9823bb3aa' 2026-02-24T17:28:02.436 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 0cc3dbc1-48a8-421e-b65d-21d9823bb3aa 2026-02-24T17:28:02.436 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:02.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCi351pJZ/NGhAABmxEPzWVk2lwTj2IhT0fXg== 2026-02-24T17:28:02.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCi351pJZ/NGhAABmxEPzWVk2lwTj2IhT0fXg=="}' 2026-02-24T17:28:02.444 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0cc3dbc1-48a8-421e-b65d-21d9823bb3aa -i td/test-erasure-eio/0/new.json 2026-02-24T17:28:02.556 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:02.563 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:28:02.564 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCi351pJZ/NGhAABmxEPzWVk2lwTj2IhT0fXg== --osd-uuid 0cc3dbc1-48a8-421e-b65d-21d9823bb3aa 2026-02-24T17:28:02.581 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:02.582+0000 7f0f24577a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:02.582 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:02.582+0000 7f0f24577a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:02.583 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:02.586+0000 7f0f24577a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:02.584 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:02.586+0000 7f0f24577a40 -1 bdev(0x561bc02b2400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:02.584 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:02.586+0000 7f0f24577a40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:28:03.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:28:03.497 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:03.497 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:28:03.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:28:03.498 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:03.612 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:28:03.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:28:03.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:03.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:03.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:03.613 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:03.629 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:03.630+0000 7f6843933a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:03.630 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:03.634+0000 7f6843933a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:03.631 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:03.634+0000 7f6843933a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:03.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:28:03.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:03.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:28:03.825 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:03.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:03.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:03.826 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:03.826 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:03.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:03.827 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:28:04.000 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:04.002+0000 7f6843933a40 -1 Falling back to public interface 2026-02-24T17:28:04.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:04.648 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:04.650+0000 7f6843933a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:28:05.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:05.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:05.046 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:05.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:05.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:05.047 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:28:05.264 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4240400754,v1:127.0.0.1:6803/4240400754] [v2:127.0.0.1:6804/4240400754,v1:127.0.0.1:6805/4240400754] exists,up 0cc3dbc1-48a8-421e-b65d-21d9823bb3aa 2026-02-24T17:28:05.268 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:28:05.269 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:05.270 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:05.271 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:28:05.272 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=deb26e42-b09b-4513-ade1-a9150f425087 2026-02-24T17:28:05.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 deb26e42-b09b-4513-ade1-a9150f425087' 2026-02-24T17:28:05.273 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 deb26e42-b09b-4513-ade1-a9150f425087 2026-02-24T17:28:05.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:05.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCl351pFggQERAAhqmdCA87xI5lU/UHnJYHYw== 2026-02-24T17:28:05.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCl351pFggQERAAhqmdCA87xI5lU/UHnJYHYw=="}' 2026-02-24T17:28:05.281 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new deb26e42-b09b-4513-ade1-a9150f425087 -i td/test-erasure-eio/1/new.json 2026-02-24T17:28:05.501 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:05.512 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:28:05.513 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCl351pFggQERAAhqmdCA87xI5lU/UHnJYHYw== --osd-uuid deb26e42-b09b-4513-ade1-a9150f425087 2026-02-24T17:28:05.539 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:05.542+0000 7f90323cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:05.541 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:05.542+0000 7f90323cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:05.542 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:05.542+0000 7f90323cda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:05.542 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:05.546+0000 7f90323cda40 -1 bdev(0x557e043a6400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:05.543 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:05.546+0000 7f90323cda40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:28:06.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:28:06.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:06.171 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:28:06.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:28:06.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:06.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:28:06.469 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:28:06.469 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:06.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:06.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:06.470 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:06.493 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:06.494+0000 7f9a95a32a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:06.495 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:06.498+0000 7f9a95a32a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:06.496 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:06.498+0000 7f9a95a32a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:06.687 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:28:06.904 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:06.948 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:06.950+0000 7f9a95a32a40 -1 Falling back to public interface 2026-02-24T17:28:07.632 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:07.634+0000 7f9a95a32a40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:28:07.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:07.905 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:07.906 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:07.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:07.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:07.907 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:28:08.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:09.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:09.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:09.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:28:09.129 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:28:09.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:09.129 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:28:09.346 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3198321760,v1:127.0.0.1:6811/3198321760] [v2:127.0.0.1:6812/3198321760,v1:127.0.0.1:6813/3198321760] exists,up deb26e42-b09b-4513-ade1-a9150f425087 2026-02-24T17:28:09.353 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:09.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:09.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:09.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:09.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:28:09.354 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:09.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:09.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:28:09.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:09.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:28:09.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:09.355 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:09.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:09.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:09.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:28:09.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:28:09.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:09.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:09.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:09.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:09.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:09.357 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:09.357 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:09.357 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:09.357 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:09.358 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:09.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:09.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:09.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:09.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:09.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:09.358 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:09.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:09.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:09.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:09.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:09.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:09.359 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:09.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:28:09.360 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:09.360 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=450038d7-ff7b-4393-bd31-cb3e33de277d 2026-02-24T17:28:09.361 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 450038d7-ff7b-4393-bd31-cb3e33de277d 2026-02-24T17:28:09.361 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 450038d7-ff7b-4393-bd31-cb3e33de277d' 2026-02-24T17:28:09.362 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:09.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCp351p7AoIFhAAukraWJ3Ms+7g8G9GhSqtWQ== 2026-02-24T17:28:09.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCp351p7AoIFhAAukraWJ3Ms+7g8G9GhSqtWQ=="}' 2026-02-24T17:28:09.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 450038d7-ff7b-4393-bd31-cb3e33de277d -i td/test-erasure-eio/2/new.json 2026-02-24T17:28:09.590 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:28:09.600 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:28:09.601 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCp351p7AoIFhAAukraWJ3Ms+7g8G9GhSqtWQ== --osd-uuid 450038d7-ff7b-4393-bd31-cb3e33de277d 2026-02-24T17:28:09.617 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:09.617+0000 7f63973f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:09.619 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:09.621+0000 7f63973f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:09.620 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:09.621+0000 7f63973f1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:09.620 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:09.621+0000 7f63973f1a40 -1 bdev(0x561eaacaa400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:09.620 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:09.621+0000 7f63973f1a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:28:10.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:28:10.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:10.242 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:28:10.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:28:10.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:10.535 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:28:10.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:28:10.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:10.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:10.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:10.536 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:10.552 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:10.553+0000 7fa81dcb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:10.553 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:10.553+0000 7fa81dcb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:10.554 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:10.553+0000 7fa81dcb7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:10.743 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:10.920 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:10.921+0000 7fa81dcb7a40 -1 Falling back to public interface 2026-02-24T17:28:10.960 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:11.496 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:11.497+0000 7fa81dcb7a40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:28:11.961 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:11.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:11.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:11.962 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:11.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:11.963 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:12.169 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1102701998,v1:127.0.0.1:6819/1102701998] [v2:127.0.0.1:6820/1102701998,v1:127.0.0.1:6821/1102701998] exists,up 450038d7-ff7b-4393-bd31-cb3e33de277d 2026-02-24T17:28:12.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:12.174 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:12.175 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:12.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:12.177 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f40b9ffe-5b40-437f-ad35-1636b26d7947 2026-02-24T17:28:12.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 f40b9ffe-5b40-437f-ad35-1636b26d7947' 2026-02-24T17:28:12.178 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 f40b9ffe-5b40-437f-ad35-1636b26d7947 2026-02-24T17:28:12.178 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:12.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCs351pzRpZCxAATXA6ow5dPGs5wLoAaNy2Ng== 2026-02-24T17:28:12.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCs351pzRpZCxAATXA6ow5dPGs5wLoAaNy2Ng=="}' 2026-02-24T17:28:12.185 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f40b9ffe-5b40-437f-ad35-1636b26d7947 -i td/test-erasure-eio/3/new.json 2026-02-24T17:28:12.407 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:28:12.416 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:28:12.417 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCs351pzRpZCxAATXA6ow5dPGs5wLoAaNy2Ng== --osd-uuid f40b9ffe-5b40-437f-ad35-1636b26d7947 2026-02-24T17:28:12.433 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:12.433+0000 7f6dca4eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:12.435 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:12.437+0000 7f6dca4eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:12.436 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:12.437+0000 7f6dca4eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:12.437 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:12.437+0000 7f6dca4eea40 -1 bdev(0x55e41a0be400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:12.437 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:12.437+0000 7f6dca4eea40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:28:13.125 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:28:13.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:13.126 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:28:13.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:28:13.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:13.424 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:28:13.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:28:13.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:13.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:13.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:13.425 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:13.448 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:13.449+0000 7fcee8d95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:13.450 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:13.449+0000 7fcee8d95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:13.451 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:13.453+0000 7fcee8d95a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:13.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:28:13.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:13.633 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:28:13.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:13.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:13.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:13.634 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:13.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:13.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:13.635 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:28:13.849 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:13.889 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:13.889+0000 7fcee8d95a40 -1 Falling back to public interface 2026-02-24T17:28:14.477 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:14.477+0000 7fcee8d95a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:28:14.850 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:14.851 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:14.851 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:14.851 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:14.851 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:14.851 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:28:15.061 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:16.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:16.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:16.062 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:28:16.063 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:28:16.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:16.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:28:16.272 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2209846590,v1:127.0.0.1:6827/2209846590] [v2:127.0.0.1:6828/2209846590,v1:127.0.0.1:6829/2209846590] exists,up f40b9ffe-5b40-437f-ad35-1636b26d7947 2026-02-24T17:28:16.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:16.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:16.272 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:16.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:28:16.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:28:16.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:28:16.273 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:16.273 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:16.273 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:16.273 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:28:16.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:28:16.274 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:28:16.336 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:28:16.342 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:28:16.344 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:28:04.002+0000 7f6843933a40 0 load: jerasure load: lrc 2026-02-24T17:28:16.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-24T17:28:16.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-24T17:28:16.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:28:16.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:28:16.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-24T17:28:16.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:28:16.345 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-24T17:28:16.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:28:16.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-24T17:28:16.614 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:28:16.615 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:28:16.982 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:28:16.991 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:28:17.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:28:17.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:28:17.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:28:17.993 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:28:17.994 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:28:17.994 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:28:17.995 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:28:17.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:28:17.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:28:17.995 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:28:18.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:28:18.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:28:18.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:28:18.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:28:18.103 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:28:18.103 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:28:18.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:28:18.307 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:18.307 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:28:18.307 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:28:18.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:28:18.307 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:18.307 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:28:18.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803781 2026-02-24T17:28:18.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803781 2026-02-24T17:28:18.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781' 2026-02-24T17:28:18.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:18.389 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:28:18.467 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607556 2026-02-24T17:28:18.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607556 2026-02-24T17:28:18.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556' 2026-02-24T17:28:18.468 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:18.468 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:28:18.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=73014444036 2026-02-24T17:28:18.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 73014444036 2026-02-24T17:28:18.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444036' 2026-02-24T17:28:18.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:18.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:28:18.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182403 2026-02-24T17:28:18.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182403 2026-02-24T17:28:18.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803781 1-51539607556 2-73014444036 3-107374182403' 2026-02-24T17:28:18.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:18.631 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803781 2026-02-24T17:28:18.631 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:18.632 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:28:18.632 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803781 2026-02-24T17:28:18.633 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:18.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803781 2026-02-24T17:28:18.634 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-24T17:28:18.634 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803781 2026-02-24T17:28:18.634 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:28:18.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-24T17:28:18.844 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:28:19.845 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:28:19.846 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:28:20.063 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-24T17:28:20.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:20.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607556 2026-02-24T17:28:20.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:20.066 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:28:20.066 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607556 2026-02-24T17:28:20.067 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:20.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607556 2026-02-24T17:28:20.068 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-24T17:28:20.068 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607556 2026-02-24T17:28:20.069 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:28:20.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607557 -lt 51539607556 2026-02-24T17:28:20.290 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:20.290 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-73014444036 2026-02-24T17:28:20.290 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:20.291 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:28:20.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-73014444036 2026-02-24T17:28:20.292 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:20.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=73014444036 2026-02-24T17:28:20.294 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 73014444036 2026-02-24T17:28:20.294 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 73014444036' 2026-02-24T17:28:20.294 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:20.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 73014444036 -lt 73014444036 2026-02-24T17:28:20.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:20.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182403 2026-02-24T17:28:20.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:20.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:28:20.522 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182403 2026-02-24T17:28:20.522 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:20.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182403 2026-02-24T17:28:20.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-24T17:28:20.524 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182403 2026-02-24T17:28:20.525 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:28:20.739 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-24T17:28:20.740 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:28:20.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:20.741 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:21.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:28:21.028 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:28:21.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:28:21.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:28:21.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:28:21.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:28:21.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:28:21.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:28:21.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:28:21.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:28:21.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:21.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-73729-0 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-73729-0 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-73729-0 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:28:21.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:21.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:28:21.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-73729-0 td/test-erasure-eio/ORIGINAL 2026-02-24T17:28:21.543 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-73729-0 td/test-erasure-eio 0 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-73729-0 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=0 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:28:21.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-73729-0 2026-02-24T17:28:21.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:28:21.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-0 2026-02-24T17:28:21.545 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-0 2026-02-24T17:28:21.545 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:28:21.763 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:28:21.763 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:21.764 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:28:21.764 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:28:21.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-24T17:28:21.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:28:21.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=3 2026-02-24T17:28:21.764 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:28:21.765 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/3/type 2026-02-24T17:28:21.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:28:21.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-24T17:28:21.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:28:21.765 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-24T17:28:21.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:28:21.766 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:28:21.767 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:28:21.767 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-24T17:28:21.767 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:28:21.767 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:28:21.767 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:21.767 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:21.767 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:21.768 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:28:21.768 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:28:21.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:28:21.828 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:28:21.829 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:28:21.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.3 2026-02-24T17:28:21.829 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-24T17:28:21.830 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-24T17:28:21.831 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:21.831 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:21.831 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:21.831 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.3.asok 2026-02-24T17:28:21.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:28:21.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-73729-0 0 2026-02-24T17:28:21.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-73729-0 2026-02-24T17:28:21.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:28:21.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:28:21.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-73729-0 2026-02-24T17:28:21.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:28:21.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:28:21.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-73729-0 td/test-erasure-eio/COPY 2026-02-24T17:28:21.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:28:21.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:28:21.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-24T17:28:21.912 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-73729-0 2026-02-24T17:28:21.912 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:28:21.912 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-0 2026-02-24T17:28:21.913 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-0 2026-02-24T17:28:21.913 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:28:22.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:28:22.129 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:22.130 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:22.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:22.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:22.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:22.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:22.440 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-24T17:28:22.692 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already out. 2026-02-24T17:28:22.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-73729-0 2026-02-24T17:28:22.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:28:22.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-0 2026-02-24T17:28:22.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-24T17:28:22.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-0 2026-02-24T17:28:22.703 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:28:22.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:28:22.925 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:22.925 INFO:tasks.workunit.client.0.trial066.stderr:0' 2026-02-24T17:28:22.925 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-24T17:28:22.926 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-24T17:28:23.186 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already in. 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:28:23.197 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:23.198 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:28:23.198 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:23.198 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:23.198 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:23.198 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:23.198 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:23.198 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:28:23.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:28:23.200 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:28:23.200 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:28:23.201 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:23.201 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:28:23.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:28:23.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:28:23.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:28:23.202 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:28:23.224 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:23.225+0000 7f3384815a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:23.226 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:23.225+0000 7f3384815a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:23.227 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:23.229+0000 7f3384815a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:23.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:28:23.411 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:23.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:28:23.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:23.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:23.412 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:23.412 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:23.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:23.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:23.413 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:23.628 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:23.693 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:23.693+0000 7f3384815a40 -1 Falling back to public interface 2026-02-24T17:28:24.320 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:24.321+0000 7f3384815a40 -1 osd.2 32 log_to_monitors true 2026-02-24T17:28:24.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:24.630 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:24.630 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:24.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:24.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:24.631 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:24.839 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 41 up_thru 17 down_at 33 last_clean_interval [17,32) [v2:127.0.0.1:6818/2622169064,v1:127.0.0.1:6819/2622169064] [v2:127.0.0.1:6820/2622169064,v1:127.0.0.1:6821/2622169064] exists,up 450038d7-ff7b-4393-bd31-cb3e33de277d 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:28:24.840 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:28:24.841 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:28:24.841 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:28:24.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:28:24.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:28:24.841 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:28:24.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:28:24.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:28:24.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:28:24.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:28:24.946 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:28:24.947 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:28:25.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:28:25.163 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:25.163 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:28:25.163 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:28:25.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:28:25.163 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:25.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:28:25.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803784 2026-02-24T17:28:25.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803784 2026-02-24T17:28:25.241 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784' 2026-02-24T17:28:25.242 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:25.242 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:28:25.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=51539607560 2026-02-24T17:28:25.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 51539607560 2026-02-24T17:28:25.323 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607560' 2026-02-24T17:28:25.324 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:25.324 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:28:25.401 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=176093659139 2026-02-24T17:28:25.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 176093659139 2026-02-24T17:28:25.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-176093659139' 2026-02-24T17:28:25.402 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:25.402 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:28:25.480 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=107374182406 2026-02-24T17:28:25.480 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 107374182406 2026-02-24T17:28:25.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803784 1-51539607560 2-176093659139 3-107374182406' 2026-02-24T17:28:25.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:25.481 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803784 2026-02-24T17:28:25.481 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:25.482 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:28:25.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803784 2026-02-24T17:28:25.482 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:25.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803784 2026-02-24T17:28:25.484 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-24T17:28:25.485 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803784 2026-02-24T17:28:25.485 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:28:25.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-24T17:28:25.702 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:25.702 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-51539607560 2026-02-24T17:28:25.702 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:25.704 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:28:25.705 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-51539607560 2026-02-24T17:28:25.705 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:25.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=51539607560 2026-02-24T17:28:25.706 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 51539607560 2026-02-24T17:28:25.706 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 51539607560' 2026-02-24T17:28:25.706 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:28:25.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 51539607560 -lt 51539607560 2026-02-24T17:28:25.929 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:25.930 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-176093659139 2026-02-24T17:28:25.930 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:25.931 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:28:25.932 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-176093659139 2026-02-24T17:28:25.932 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:25.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=176093659139 2026-02-24T17:28:25.934 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 176093659139 2026-02-24T17:28:25.934 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 176093659139' 2026-02-24T17:28:25.934 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:26.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 176093659139 -lt 176093659139 2026-02-24T17:28:26.160 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:26.161 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-107374182406 2026-02-24T17:28:26.161 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:26.162 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:28:26.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-107374182406 2026-02-24T17:28:26.163 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:26.164 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=107374182406 2026-02-24T17:28:26.164 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 107374182406 2026-02-24T17:28:26.165 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 107374182406' 2026-02-24T17:28:26.165 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:28:26.382 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-24T17:28:26.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:28:26.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:26.383 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:26.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:28:26.670 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:28:26.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:28:26.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:28:26.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:28:26.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:28:26.670 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:28:26.671 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:28:26.884 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:28:26.884 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:28:26.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:26.885 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:27.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:28:27.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:28:27.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:28:27.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:28:27.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:28:27.170 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:28:27.373 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:28:27.383 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:28:27.682 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:28:27.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:28:27.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:28:27.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:28:27.694 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:28:27.695 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:27.695 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:27.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:27.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:27.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:28.007 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:28.007 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:28:28.008 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:28:28.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:28:28.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:28:28.011 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:28:28.011 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:28:28.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:28.012 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:28:28.013 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:28:28.013 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:28.014 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:28:28.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:28:28.016 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:28:28.030 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:28:28.030 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:28.030 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:28.030 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:28:28.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:28:28.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:28:28.032 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:28.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:28.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:28.035 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:28:28.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:28:28.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:28:28.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:28:28.038 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:28:28.038 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:28:28.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:28.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:28:28.040 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:28:28.041 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:28.041 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:28:28.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:28:28.043 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:28:28.044 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:28:28.045 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:28.045 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:28.045 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:28:28.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:28:28.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:28:28.046 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-24T17:28:28.047 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-24T17:28:28.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:28.048 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:28.048 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.73729 2026-02-24T17:28:28.049 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-24T17:28:28.051 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-24T17:28:28.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-24T17:28:28.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:28.072 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:28.073 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:28.073 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:28.073 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:28.073 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:28.073 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-24T17:28:28.110 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-24T17:28:28.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-24T17:28:28.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:28:28.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:28:28.111 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-24T17:28:28.113 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-24T17:28:28.113 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:28:28.113 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:28:28.113 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:28:28.113 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:28.113 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:28.114 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:28.114 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:28:28.114 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:28:28.114 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get fsid 2026-02-24T17:28:28.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-24T17:28:28.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-24T17:28:28.169 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-24T17:28:28.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-24T17:28:28.170 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-24T17:28:28.171 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-24T17:28:28.171 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:28:28.171 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:28:28.171 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:28.171 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:28.171 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:28.172 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:28:28.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-24T17:28:28.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.73729/ceph-mon.a.asok config get mon_host 2026-02-24T17:28:28.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-24T17:28:28.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-24T17:28:28.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-24T17:28:28.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-24T17:28:28.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-24T17:28:28.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-24T17:28:28.228 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-24T17:28:28.344 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-24T17:28:28.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:28.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:28.345 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:28.345 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:28.345 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:28.345 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:28.346 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:28:28.346 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-24T17:28:28.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-24T17:28:28.364 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-24T17:28:28.470 INFO:tasks.workunit.client.0.trial066.stderr:pool 'rbd' already exists 2026-02-24T17:28:28.478 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:28:29.480 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-24T17:28:29.480 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-24T17:28:29.480 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-24T17:28:29.480 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:29.480 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:29.480 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:29.481 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-mon.a.asok 2026-02-24T17:28:29.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-24T17:28:29.481 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-mon.a.asok log flush 2026-02-24T17:28:29.531 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:28:29.537 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-24T17:28:29.539 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:28:28.104+0000 7f277741dec0 0 load: jerasure load: lrc 2026-02-24T17:28:29.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-24T17:28:29.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-24T17:28:29.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-24T17:28:29.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-24T17:28:29.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-24T17:28:29.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-24T17:28:29.540 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-24T17:28:29.541 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-24T17:28:29.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:29.548 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:29.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=aa739c28-c982-49a6-aa35-5cc1db98bf07 2026-02-24T17:28:29.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 aa739c28-c982-49a6-aa35-5cc1db98bf07' 2026-02-24T17:28:29.551 INFO:tasks.workunit.client.0.trial066.stdout:add osd0 aa739c28-c982-49a6-aa35-5cc1db98bf07 2026-02-24T17:28:29.551 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:29.557 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC9351puth8IRAApvkxxzD90G7YmDbUesaMpw== 2026-02-24T17:28:29.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC9351puth8IRAApvkxxzD90G7YmDbUesaMpw=="}' 2026-02-24T17:28:29.558 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new aa739c28-c982-49a6-aa35-5cc1db98bf07 -i td/test-erasure-eio/0/new.json 2026-02-24T17:28:29.673 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:29.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-24T17:28:29.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC9351puth8IRAApvkxxzD90G7YmDbUesaMpw== --osd-uuid aa739c28-c982-49a6-aa35-5cc1db98bf07 2026-02-24T17:28:29.698 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:29.700+0000 7f29754caa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:29.700 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:29.704+0000 7f29754caa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:29.701 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:29.704+0000 7f29754caa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:29.701 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:29.704+0000 7f29754caa40 -1 bdev(0x55e5e80c8400 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:29.701 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:29.704+0000 7f29754caa40 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-24T17:28:30.441 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-24T17:28:30.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:30.442 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-24T17:28:30.443 INFO:tasks.workunit.client.0.trial066.stdout:adding osd0 key to auth repository 2026-02-24T17:28:30.443 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:30.585 INFO:tasks.workunit.client.0.trial066.stdout:start osd.0 2026-02-24T17:28:30.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-24T17:28:30.586 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:30.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:30.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:30.587 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:30.600 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:30.604+0000 7f2e2d5e8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:30.601 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:30.604+0000 7f2e2d5e8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:30.602 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:30.604+0000 7f2e2d5e8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:30.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-24T17:28:30.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:30.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-24T17:28:30.698 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:30.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:30.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:30.699 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:30.699 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:30.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:30.700 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:28:30.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:30.997 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:31.000+0000 7f2e2d5e8a40 -1 Falling back to public interface 2026-02-24T17:28:31.556 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:31.560+0000 7f2e2d5e8a40 -1 osd.0 0 log_to_monitors true 2026-02-24T17:28:31.909 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:31.910 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:31.910 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:31.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:31.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:31.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:28:32.121 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:33.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:33.123 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:33.123 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:28:33.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:28:33.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:33.124 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-24T17:28:33.334 INFO:tasks.workunit.client.0.trial066.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1010562994,v1:127.0.0.1:6803/1010562994] [v2:127.0.0.1:6804/1010562994,v1:127.0.0.1:6805/1010562994] exists,up aa739c28-c982-49a6-aa35-5cc1db98bf07 2026-02-24T17:28:33.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:33.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:33.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:33.335 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-24T17:28:33.336 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-24T17:28:33.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:33.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:33.337 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:33.337 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:33.337 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:33.337 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:33.339 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:33.339 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:33.339 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:33.339 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:33.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:33.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:33.339 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:33.340 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3051a463-c069-4e0f-b272-91e7847eafe7 2026-02-24T17:28:33.341 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 3051a463-c069-4e0f-b272-91e7847eafe7' 2026-02-24T17:28:33.341 INFO:tasks.workunit.client.0.trial066.stdout:add osd1 3051a463-c069-4e0f-b272-91e7847eafe7 2026-02-24T17:28:33.341 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:33.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDB351plWsxFRAALKs0rnpRIHBCFqruB3wm0A== 2026-02-24T17:28:33.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDB351plWsxFRAALKs0rnpRIHBCFqruB3wm0A=="}' 2026-02-24T17:28:33.351 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3051a463-c069-4e0f-b272-91e7847eafe7 -i td/test-erasure-eio/1/new.json 2026-02-24T17:28:33.568 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:33.578 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-24T17:28:33.579 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDB351plWsxFRAALKs0rnpRIHBCFqruB3wm0A== --osd-uuid 3051a463-c069-4e0f-b272-91e7847eafe7 2026-02-24T17:28:33.596 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:33.600+0000 7f1cbcc18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:33.597 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:33.600+0000 7f1cbcc18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:33.598 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:33.600+0000 7f1cbcc18a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:33.599 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:33.600+0000 7f1cbcc18a40 -1 bdev(0x55da7a85a400 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:33.599 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:33.600+0000 7f1cbcc18a40 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-24T17:28:34.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-24T17:28:34.243 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:34.244 INFO:tasks.workunit.client.0.trial066.stdout:adding osd1 key to auth repository 2026-02-24T17:28:34.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-24T17:28:34.244 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:34.524 INFO:tasks.workunit.client.0.trial066.stdout:start osd.1 2026-02-24T17:28:34.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-24T17:28:34.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:34.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:34.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:34.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:34.547 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:34.548+0000 7f50d1c6da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:34.549 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:34.552+0000 7f50d1c6da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:34.550 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:34.552+0000 7f50d1c6da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:34.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-24T17:28:34.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:34.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-24T17:28:34.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:34.745 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:34.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:34.746 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:34.746 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:34.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:34.747 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:28:34.941 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:34.944+0000 7f50d1c6da40 -1 Falling back to public interface 2026-02-24T17:28:34.962 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:35.673 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:35.676+0000 7f50d1c6da40 -1 osd.1 0 log_to_monitors true 2026-02-24T17:28:35.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:35.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:35.964 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:35.965 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:35.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:35.965 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:28:36.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:37.178 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:37.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:37.179 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:28:37.179 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:28:37.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:37.180 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-24T17:28:37.391 INFO:tasks.workunit.client.0.trial066.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1239738556,v1:127.0.0.1:6811/1239738556] [v2:127.0.0.1:6812/1239738556,v1:127.0.0.1:6813/1239738556] exists,up 3051a463-c069-4e0f-b272-91e7847eafe7 2026-02-24T17:28:37.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:37.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:37.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:37.395 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:28:37.396 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:37.397 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:28:37.398 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:37.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b2eab976-5500-44ef-a871-ba072a24783a 2026-02-24T17:28:37.399 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 b2eab976-5500-44ef-a871-ba072a24783a' 2026-02-24T17:28:37.400 INFO:tasks.workunit.client.0.trial066.stdout:add osd2 b2eab976-5500-44ef-a871-ba072a24783a 2026-02-24T17:28:37.400 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:37.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDF351pFsiiGBAAGm/nU5RCuMy6kvTe6Wij+g== 2026-02-24T17:28:37.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDF351pFsiiGBAAGm/nU5RCuMy6kvTe6Wij+g=="}' 2026-02-24T17:28:37.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b2eab976-5500-44ef-a871-ba072a24783a -i td/test-erasure-eio/2/new.json 2026-02-24T17:28:37.627 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:28:37.636 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-24T17:28:37.637 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDF351pFsiiGBAAGm/nU5RCuMy6kvTe6Wij+g== --osd-uuid b2eab976-5500-44ef-a871-ba072a24783a 2026-02-24T17:28:37.653 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:37.656+0000 7fd71dc50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:37.655 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:37.656+0000 7fd71dc50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:37.656 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:37.660+0000 7fd71dc50a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:37.656 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:37.660+0000 7fd71dc50a40 -1 bdev(0x558183302400 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:37.656 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:37.660+0000 7fd71dc50a40 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-24T17:28:38.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-24T17:28:38.278 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:38.279 INFO:tasks.workunit.client.0.trial066.stdout:adding osd2 key to auth repository 2026-02-24T17:28:38.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-24T17:28:38.280 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:38.559 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:28:38.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-24T17:28:38.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:38.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:38.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:38.559 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:38.583 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:38.584+0000 7f8f9c68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:38.585 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:38.588+0000 7f8f9c68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:38.586 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:38.588+0000 7f8f9c68ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:38.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-24T17:28:38.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:38.779 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:28:38.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:38.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:38.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:38.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:38.780 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:38.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:38.780 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:38.953 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:38.956+0000 7f8f9c68ea40 -1 Falling back to public interface 2026-02-24T17:28:38.998 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:39.561 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:39.564+0000 7f8f9c68ea40 -1 osd.2 0 log_to_monitors true 2026-02-24T17:28:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:39.999 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:39.999 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:40.215 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3271405579,v1:127.0.0.1:6819/3271405579] [v2:127.0.0.1:6820/3271405579,v1:127.0.0.1:6821/3271405579] exists,up b2eab976-5500-44ef-a871-ba072a24783a 2026-02-24T17:28:40.218 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-24T17:28:40.219 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:40.220 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:40.221 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:40.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-24T17:28:40.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-24T17:28:40.222 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-24T17:28:40.222 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-24T17:28:40.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=779b9e44-dceb-45a9-a464-b7e5460bbcac 2026-02-24T17:28:40.223 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 779b9e44-dceb-45a9-a464-b7e5460bbcac' 2026-02-24T17:28:40.223 INFO:tasks.workunit.client.0.trial066.stdout:add osd3 779b9e44-dceb-45a9-a464-b7e5460bbcac 2026-02-24T17:28:40.223 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-24T17:28:40.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDI351pMgUiDhAAzgtFEfmVIdrEPNQQL2bMyQ== 2026-02-24T17:28:40.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDI351pMgUiDhAAzgtFEfmVIdrEPNQQL2bMyQ=="}' 2026-02-24T17:28:40.233 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 779b9e44-dceb-45a9-a464-b7e5460bbcac -i td/test-erasure-eio/3/new.json 2026-02-24T17:28:40.454 INFO:tasks.workunit.client.0.trial066.stdout:3 2026-02-24T17:28:40.463 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-24T17:28:40.464 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDI351pMgUiDhAAzgtFEfmVIdrEPNQQL2bMyQ== --osd-uuid 779b9e44-dceb-45a9-a464-b7e5460bbcac 2026-02-24T17:28:40.481 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:40.484+0000 7f45d4b59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:40.483 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:40.484+0000 7f45d4b59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:40.484 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:40.484+0000 7f45d4b59a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:40.484 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:40.488+0000 7f45d4b59a40 -1 bdev(0x559d4c748400 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-24T17:28:40.484 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:40.488+0000 7f45d4b59a40 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-24T17:28:41.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-24T17:28:41.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-24T17:28:41.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-24T17:28:41.172 INFO:tasks.workunit.client.0.trial066.stdout:adding osd3 key to auth repository 2026-02-24T17:28:41.172 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-24T17:28:41.460 INFO:tasks.workunit.client.0.trial066.stdout:start osd.3 2026-02-24T17:28:41.461 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-24T17:28:41.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:41.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-24T17:28:41.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-24T17:28:41.462 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-24T17:28:41.475 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:41.476+0000 7feb1a9f4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:41.476 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:41.480+0000 7feb1a9f4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:41.477 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:41.480+0000 7feb1a9f4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-24T17:28:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-24T17:28:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:41.680 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:41.681 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:41.681 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:41.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:41.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:28:41.888 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:41.902 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:41.904+0000 7feb1a9f4a40 -1 Falling back to public interface 2026-02-24T17:28:42.476 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:42.480+0000 7feb1a9f4a40 -1 osd.3 0 log_to_monitors true 2026-02-24T17:28:42.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:42.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:42.889 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:42.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:42.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:42.890 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:28:43.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:44.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:44.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:44.101 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-24T17:28:44.101 INFO:tasks.workunit.client.0.trial066.stdout:2 2026-02-24T17:28:44.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:44.102 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-24T17:28:44.308 INFO:tasks.workunit.client.0.trial066.stdout:osd.3 up in weight 1 up_from 26 up_thru 29 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1317330469,v1:127.0.0.1:6827/1317330469] [v2:127.0.0.1:6828/1317330469,v1:127.0.0.1:6829/1317330469] exists,up 779b9e44-dceb-45a9-a464-b7e5460bbcac 2026-02-24T17:28:44.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:44.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:44.308 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:44.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-24T17:28:44.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-24T17:28:44.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-24T17:28:44.309 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:44.309 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:44.309 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:44.309 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.0.asok 2026-02-24T17:28:44.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-24T17:28:44.310 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.0.asok log flush 2026-02-24T17:28:44.372 INFO:tasks.workunit.client.0.trial066.stdout:{} 2026-02-24T17:28:44.378 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-24T17:28:44.379 INFO:tasks.workunit.client.0.trial066.stdout:2026-02-24T17:28:31.004+0000 7f2e2d5e8a40 0 load: jerasure load: lrc 2026-02-24T17:28:44.380 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-24T17:28:44.381 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-24T17:28:44.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-24T17:28:44.651 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-24T17:28:45.022 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' already exists 2026-02-24T17:28:45.033 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-24T17:28:46.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-24T17:28:46.035 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:28:46.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:28:46.036 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:28:46.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:28:46.037 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:28:46.037 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:28:46.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:28:46.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:28:46.037 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:28:46.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:28:46.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:28:46.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:28:46.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:28:46.145 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:28:46.145 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:28:46.365 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:28:46.365 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:46.366 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:28:46.366 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:28:46.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:28:46.366 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:46.366 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:28:46.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803782 2026-02-24T17:28:46.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803782 2026-02-24T17:28:46.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782' 2026-02-24T17:28:46.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:46.448 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:28:46.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574852 2026-02-24T17:28:46.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574852 2026-02-24T17:28:46.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-55834574852' 2026-02-24T17:28:46.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:46.525 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:28:46.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=77309411333 2026-02-24T17:28:46.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 77309411333 2026-02-24T17:28:46.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-55834574852 2-77309411333' 2026-02-24T17:28:46.605 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:46.605 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:28:46.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149699 2026-02-24T17:28:46.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149699 2026-02-24T17:28:46.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803782 1-55834574852 2-77309411333 3-111669149699' 2026-02-24T17:28:46.682 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:46.682 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803782 2026-02-24T17:28:46.682 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:46.684 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:28:46.684 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803782 2026-02-24T17:28:46.684 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:46.686 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803782 2026-02-24T17:28:46.686 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803782 2026-02-24T17:28:46.686 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-24T17:28:46.686 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:28:46.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-24T17:28:46.906 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:46.907 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574852 2026-02-24T17:28:46.907 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:46.908 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:28:46.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574852 2026-02-24T17:28:46.909 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:46.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574852 2026-02-24T17:28:46.911 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-24T17:28:46.911 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 55834574852 2026-02-24T17:28:46.912 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:28:47.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-24T17:28:47.127 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:47.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-77309411333 2026-02-24T17:28:47.127 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:47.128 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:28:47.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-77309411333 2026-02-24T17:28:47.129 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:47.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=77309411333 2026-02-24T17:28:47.131 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-24T17:28:47.131 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 77309411333 2026-02-24T17:28:47.131 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:47.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411331 -lt 77309411333 2026-02-24T17:28:47.356 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:28:48.357 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:28:48.358 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:48.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411331 -lt 77309411333 2026-02-24T17:28:48.571 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:28:49.572 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:28:49.573 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:49.792 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-24T17:28:49.793 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:49.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149699 2026-02-24T17:28:49.793 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:49.795 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:28:49.796 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149699 2026-02-24T17:28:49.796 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:49.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149699 2026-02-24T17:28:49.797 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 111669149699 2026-02-24T17:28:49.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-24T17:28:49.798 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:28:50.009 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-24T17:28:50.009 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:28:50.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:50.010 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:50.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:28:50.289 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:28:50.290 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:28:50.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:28:50.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:28:50.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:28:50.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:28:50.291 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:28:50.500 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:28:50.501 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:28:50.501 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:50.502 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:50.796 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-24T17:28:50.797 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-73729-1 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-73729-1 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-73729-1 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:50.798 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-24T17:28:50.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:50.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-24T17:28:50.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-24T17:28:50.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-24T17:28:50.799 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-73729-1 td/test-erasure-eio/ORIGINAL 2026-02-24T17:28:50.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-73729-1 td/test-erasure-eio 1 2026-02-24T17:28:50.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2509: inject_eio: local pooltype=ec 2026-02-24T17:28:50.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2510: inject_eio: shift 2026-02-24T17:28:50.815 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: inject_eio: local which=data 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: inject_eio: shift 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2513: inject_eio: local poolname=pool-jerasure 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2514: inject_eio: shift 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2515: inject_eio: local objname=obj-eio-73729-1 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2516: inject_eio: shift 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2517: inject_eio: local dir=td/test-erasure-eio 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2518: inject_eio: shift 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2519: inject_eio: local shard_id=1 2026-02-24T17:28:50.816 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2520: inject_eio: shift 2026-02-24T17:28:50.817 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: get_osds pool-jerasure obj-eio-73729-1 2026-02-24T17:28:50.817 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:28:50.817 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-1 2026-02-24T17:28:50.817 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-1 2026-02-24T17:28:50.817 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:28:51.025 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:28:51.025 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:51.026 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:28:51.026 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:28:51.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: initial_osds=('3' '1' '2') 2026-02-24T17:28:51.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: inject_eio: local -a initial_osds 2026-02-24T17:28:51.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: inject_eio: local osd_id=1 2026-02-24T17:28:51.026 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: inject_eio: '[' ec '!=' ec ']' 2026-02-24T17:28:51.026 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: cat td/test-erasure-eio/1/type 2026-02-24T17:28:51.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: inject_eio: type=bluestore 2026-02-24T17:28:51.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2528: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-24T17:28:51.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-24T17:28:51.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-24T17:28:51.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-24T17:28:51.027 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-24T17:28:51.028 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-24T17:28:51.028 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-24T17:28:51.028 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:28:51.028 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:28:51.029 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:51.029 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:51.029 INFO:tasks.workunit.client.0.trial066.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:51.029 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:28:51.029 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.73729/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-24T17:28:51.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-24T17:28:51.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: inject_eio: local loop=0 2026-02-24T17:28:51.092 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: inject_eio: grep -q Invalid 2026-02-24T17:28:51.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: get_asok_path osd.1 2026-02-24T17:28:51.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-24T17:28:51.093 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-24T17:28:51.093 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-24T17:28:51.093 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:51.093 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:51.094 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.73729/ceph-osd.1.asok 2026-02-24T17:28:51.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: CEPH_ARGS= 2026-02-24T17:28:51.094 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2530: inject_eio: ceph --admin-daemon /tmp/ceph-asok.73729/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-73729-1 1 2026-02-24T17:28:51.154 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-73729-1 2026-02-24T17:28:51.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-24T17:28:51.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-24T17:28:51.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-73729-1 2026-02-24T17:28:51.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-24T17:28:51.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-24T17:28:51.155 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-73729-1 td/test-erasure-eio/COPY 2026-02-24T17:28:51.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-24T17:28:51.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-24T17:28:51.176 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-24T17:28:51.177 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-73729-1 2026-02-24T17:28:51.177 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:28:51.177 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-1 2026-02-24T17:28:51.178 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-1 2026-02-24T17:28:51.178 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:28:51.387 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:28:51.387 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:51.387 INFO:tasks.workunit.client.0.trial066.stderr:2' 2026-02-24T17:28:51.387 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-24T17:28:51.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-24T17:28:51.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-24T17:28:51.387 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-24T17:28:51.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-24T17:28:51.388 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:51.388 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:51.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:51.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:51.388 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:51.695 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:51.696 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-24T17:28:51.958 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already out. 2026-02-24T17:28:51.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-73729-1 2026-02-24T17:28:51.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-24T17:28:51.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-24T17:28:51.970 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-73729-1 2026-02-24T17:28:51.971 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-73729-1 2026-02-24T17:28:51.971 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-24T17:28:52.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-24T17:28:52.189 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:52.189 INFO:tasks.workunit.client.0.trial066.stderr:0' 2026-02-24T17:28:52.189 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-24T17:28:52.190 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-24T17:28:52.433 INFO:tasks.workunit.client.0.trial066.stderr:osd.2 is already in. 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-24T17:28:52.446 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-24T17:28:52.447 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-24T17:28:52.448 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-24T17:28:52.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-24T17:28:52.449 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-24T17:28:52.449 INFO:tasks.workunit.client.0.trial066.stdout:start osd.2 2026-02-24T17:28:52.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-24T17:28:52.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=24f71ff6-82c3-44c0-aa77-0f7043a4afee --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.73729/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-24T17:28:52.450 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-24T17:28:52.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-24T17:28:52.450 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-24T17:28:52.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-24T17:28:52.451 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-24T17:28:52.470 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:52.471+0000 7fc8afc3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:52.471 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:52.471+0000 7fc8afc3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:52.472 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:52.475+0000 7fc8afc3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-24T17:28:52.666 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-24T17:28:52.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-24T17:28:52.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-24T17:28:52.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-24T17:28:52.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-24T17:28:52.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:52.667 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-24T17:28:52.667 INFO:tasks.workunit.client.0.trial066.stdout:0 2026-02-24T17:28:52.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:52.668 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:52.887 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-24T17:28:52.894 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:52.895+0000 7fc8afc3aa40 -1 Falling back to public interface 2026-02-24T17:28:53.477 INFO:tasks.workunit.client.0.trial066.stderr:2026-02-24T17:28:53.479+0000 7fc8afc3aa40 -1 osd.2 35 log_to_monitors true 2026-02-24T17:28:53.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-24T17:28:53.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-24T17:28:53.889 INFO:tasks.workunit.client.0.trial066.stdout:1 2026-02-24T17:28:53.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-24T17:28:53.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-24T17:28:53.889 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-24T17:28:54.097 INFO:tasks.workunit.client.0.trial066.stdout:osd.2 up in weight 1 up_from 44 up_thru 20 down_at 36 last_clean_interval [18,35) [v2:127.0.0.1:6818/3548013056,v1:127.0.0.1:6819/3548013056] [v2:127.0.0.1:6820/3548013056,v1:127.0.0.1:6821/3548013056] exists,up b2eab976-5500-44ef-a871-ba072a24783a 2026-02-24T17:28:54.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-24T17:28:54.099 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-24T17:28:54.100 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-24T17:28:54.101 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-24T17:28:54.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-24T17:28:54.198 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-24T17:28:54.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-24T17:28:54.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-24T17:28:54.199 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: flush_pg_stats: local timeout=300 2026-02-24T17:28:54.199 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ceph osd ls 2026-02-24T17:28:54.407 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: flush_pg_stats: ids='0 2026-02-24T17:28:54.407 INFO:tasks.workunit.client.0.trial066.stderr:1 2026-02-24T17:28:54.408 INFO:tasks.workunit.client.0.trial066.stderr:2 2026-02-24T17:28:54.408 INFO:tasks.workunit.client.0.trial066.stderr:3' 2026-02-24T17:28:54.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: flush_pg_stats: seqs= 2026-02-24T17:28:54.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:54.408 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-24T17:28:54.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=25769803786 2026-02-24T17:28:54.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 25769803786 2026-02-24T17:28:54.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786' 2026-02-24T17:28:54.488 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:54.488 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-24T17:28:54.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=55834574856 2026-02-24T17:28:54.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 55834574856 2026-02-24T17:28:54.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574856' 2026-02-24T17:28:54.566 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:54.566 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-24T17:28:54.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=188978561027 2026-02-24T17:28:54.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 188978561027 2026-02-24T17:28:54.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574856 2-188978561027' 2026-02-24T17:28:54.645 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: flush_pg_stats: for osd in $ids 2026-02-24T17:28:54.646 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-24T17:28:54.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: flush_pg_stats: seq=111669149703 2026-02-24T17:28:54.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: flush_pg_stats: test -z 111669149703 2026-02-24T17:28:54.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: flush_pg_stats: seqs=' 0-25769803786 1-55834574856 2-188978561027 3-111669149703' 2026-02-24T17:28:54.725 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:54.726 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 0-25769803786 2026-02-24T17:28:54.726 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:54.726 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=0 2026-02-24T17:28:54.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 0-25769803786 2026-02-24T17:28:54.727 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:54.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=25769803786 2026-02-24T17:28:54.729 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-24T17:28:54.729 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.0 seq 25769803786 2026-02-24T17:28:54.730 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-24T17:28:54.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-24T17:28:54.949 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:54.950 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 1-55834574856 2026-02-24T17:28:54.950 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:54.951 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=1 2026-02-24T17:28:54.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 1-55834574856 2026-02-24T17:28:54.952 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:54.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=55834574856 2026-02-24T17:28:54.953 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.1 seq 55834574856' 2026-02-24T17:28:54.954 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.1 seq 55834574856 2026-02-24T17:28:54.954 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-24T17:28:55.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-24T17:28:55.171 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:55.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 2-188978561027 2026-02-24T17:28:55.172 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:55.173 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=2 2026-02-24T17:28:55.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 2-188978561027 2026-02-24T17:28:55.174 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:55.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=188978561027 2026-02-24T17:28:55.175 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.2 seq 188978561027' 2026-02-24T17:28:55.176 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.2 seq 188978561027 2026-02-24T17:28:55.176 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:55.393 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 188978561027 2026-02-24T17:28:55.393 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:28:56.394 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-24T17:28:56.394 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:56.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 1 -lt 188978561027 2026-02-24T17:28:56.616 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2299: flush_pg_stats: sleep 1 2026-02-24T17:28:57.617 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2300: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-24T17:28:57.618 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-24T17:28:57.831 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 188978561027 -lt 188978561027 2026-02-24T17:28:57.832 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: flush_pg_stats: for s in $seqs 2026-02-24T17:28:57.832 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: echo 3-111669149703 2026-02-24T17:28:57.832 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: cut -d - -f 1 2026-02-24T17:28:57.833 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2295: flush_pg_stats: osd=3 2026-02-24T17:28:57.834 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: echo 3-111669149703 2026-02-24T17:28:57.834 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: cut -d - -f 2 2026-02-24T17:28:57.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2296: flush_pg_stats: seq=111669149703 2026-02-24T17:28:57.836 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2297: flush_pg_stats: echo 'waiting osd.3 seq 111669149703' 2026-02-24T17:28:57.836 INFO:tasks.workunit.client.0.trial066.stdout:waiting osd.3 seq 111669149703 2026-02-24T17:28:57.837 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-24T17:28:58.064 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2298: flush_pg_stats: test 111669149703 -lt 111669149703 2026-02-24T17:28:58.064 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-24T17:28:58.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:58.065 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:58.349 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-24T17:28:58.350 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-24T17:28:58.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-24T17:28:58.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-24T17:28:58.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-24T17:28:58.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-24T17:28:58.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-24T17:28:58.350 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-24T17:28:58.569 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-24T17:28:58.570 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-24T17:28:58.570 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-24T17:28:58.570 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-24T17:28:58.859 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-24T17:28:58.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-24T17:28:58.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-24T17:28:58.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-24T17:28:58.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-24T17:28:58.860 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-24T17:28:59.116 INFO:tasks.workunit.client.0.trial066.stderr:pool 'pool-jerasure' does not exist 2026-02-24T17:28:59.126 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-24T17:28:59.398 INFO:tasks.workunit.client.0.trial066.stderr:erasure-code-profile myprofile does not exist 2026-02-24T17:28:59.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-24T17:28:59.408 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:28:59.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-24T17:28:59.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:28:59.409 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:59.409 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:59.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:59.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:59.409 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:59.519 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:59.519 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:28:59.520 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:28:59.520 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:28:59.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:28:59.521 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:28:59.522 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:28:59.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:59.522 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:28:59.523 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:28:59.523 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:59.524 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:28:59.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-24T17:28:59.525 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:28:59.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:28:59.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:59.539 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:59.539 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:28:59.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:28:59.541 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2429: main: code=0 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2433: main: teardown td/test-erasure-eio 0 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-24T17:28:59.542 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-24T17:28:59.544 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-24T17:28:59.544 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-24T17:28:59.545 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-24T17:28:59.545 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-24T17:28:59.546 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-24T17:28:59.547 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-24T17:28:59.547 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-24T17:28:59.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:59.548 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-24T17:28:59.549 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-24T17:28:59.549 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T17:28:59.550 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:28:59.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-24T17:28:59.551 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-24T17:28:59.552 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-24T17:28:59.553 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-24T17:28:59.553 INFO:tasks.workunit.client.0.trial066.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.73729 2026-02-24T17:28:59.553 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.73729 2026-02-24T17:28:59.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-24T17:28:59.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-24T17:28:59.554 INFO:tasks.workunit.client.0.trial066.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2434: main: return 0 2026-02-24T17:28:59.555 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-24T17:28:59.556 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-24T17:28:59.609 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-24T17:28:59.609 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-24T17:28:59.974 DEBUG:teuthology.parallel:result is None 2026-02-24T17:28:59.974 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T17:28:59.982 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T17:28:59.982 DEBUG:teuthology.orchestra.run.trial066:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-24T17:29:00.029 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-24T17:29:00.029 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-24T17:29:00.045 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-24T17:29:00.046 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T17:29:02.145 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-24T17:29:02.160 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-24T17:29:02.160 DEBUG:teuthology.orchestra.run.trial066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T17:29:02.249 INFO:teuthology.orchestra.run.trial066.stdout: remote refid st t when poll reach delay offset jitter 2026-02-24T17:29:02.249 INFO:teuthology.orchestra.run.trial066.stdout:============================================================================== 2026-02-24T17:29:02.249 INFO:teuthology.orchestra.run.trial066.stdout:*ntp.swyn.net 207.66.79.103 2 u 44 64 377 3.587 -9.504 1.483 2026-02-24T17:29:02.249 INFO:teuthology.orchestra.run.trial066.stdout:+clock.xmission. 132.163.96.1 2 u 52 64 377 65.670 -8.134 2.442 2026-02-24T17:29:02.250 INFO:teuthology.orchestra.run.trial066.stdout:+time.cloudflare 10.25.8.4 3 u 42 64 377 3.113 -10.710 1.867 2026-02-24T17:29:02.250 INFO:teuthology.orchestra.run.trial066.stdout:-t1.time.bf1.yah 98.139.133.62 2 u 40 64 377 82.577 +24.581 1.781 2026-02-24T17:29:02.251 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-24T17:29:02.267 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-24T17:29:02.268 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-24T17:29:02.282 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-24T17:29:02.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-24T17:29:02.310 INFO:teuthology.task.internal:Duration was 1116.353003 seconds 2026-02-24T17:29:02.311 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-24T17:29:02.326 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-24T17:29:02.326 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T17:29:02.353 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-24T17:29:02.353 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial066.front.sepia.ceph.com 2026-02-24T17:29:02.354 DEBUG:teuthology.orchestra.run.trial066:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T17:29:02.403 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-24T17:29:02.403 DEBUG:teuthology.orchestra.run.trial066:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T17:29:02.670 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-24T17:29:02.670 DEBUG:teuthology.orchestra.run.trial066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T17:29:02.680 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T17:29:02.681 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T17:29:02.681 INFO:teuthology.orchestra.run.trial066.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T17:29:02.681 INFO:teuthology.orchestra.run.trial066.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T17:29:02.681 INFO:teuthology.orchestra.run.trial066.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T17:29:02.751 INFO:teuthology.orchestra.run.trial066.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T17:29:02.753 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-24T17:29:02.769 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-24T17:29:02.770 DEBUG:teuthology.orchestra.run.trial066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T17:29:02.805 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-24T17:29:02.821 DEBUG:teuthology.orchestra.run.trial066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:29:02.856 INFO:teuthology.orchestra.run.trial066.stdout:kernel.core_pattern = core 2026-02-24T17:29:02.865 DEBUG:teuthology.orchestra.run.trial066:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T17:29:02.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T17:29:02.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-24T17:29:02.925 INFO:teuthology.task.internal:Transferring archived files... 2026-02-24T17:29:02.926 DEBUG:teuthology.misc:Transferring archived files from trial066:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:03:54-rados-wip-yuri8-testing-2026-02-23-2128-distro-default-trial/68438/remote/trial066 2026-02-24T17:29:02.927 DEBUG:teuthology.orchestra.run.trial066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T17:29:03.002 INFO:teuthology.task.internal:Removing archive directory... 2026-02-24T17:29:03.002 DEBUG:teuthology.orchestra.run.trial066:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T17:29:03.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-24T17:29:03.040 INFO:teuthology.task.internal:Not uploading archives. 2026-02-24T17:29:03.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-24T17:29:03.057 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-24T17:29:03.057 DEBUG:teuthology.orchestra.run.trial066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T17:29:03.068 INFO:teuthology.orchestra.run.trial066.stdout: 1311292 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 17:29 /home/ubuntu/cephtest 2026-02-24T17:29:03.070 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-24T17:29:03.087 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-24T17:29:03.125 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} duration: 1116.3530027866364 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-24T17:29:03.125 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T17:29:03.251 INFO:teuthology.run:pass